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FOREWORD 


A New Inquiry Approach to 
High-Impact Practices 
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In 2008, the Association of American Colleges and Universities (AAC&U) published George 
Kuh’s High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They 
Matter.' The book produced a groundswell of interest in the higher education community. At 
colleges and universities across the country, faculty and educational leaders turned their attention 
to high-impact practices—or HIPs—talking about them, asking for information, trying to be 
more intentional in practice. Some people began to think about employing HIPs to boost the 
achievement of underserved students. Some institutions and higher education systems started to 
consider redesigning the student experience around HIPs and related pedagogies. Higher 
education scholars opened new lines of inquiry. The book became a timely and refreshing 
prompt—but not surprisingly, with popularity came a certain degree of controversy. People 
began to ask and debate, for example, what precise impact on whom and how much do we really know? 


Assessing Underserved Students’ Engagement in High-Impact Practices contributes to the national 
dialogue by building on Kuh’s original findings from his work with the large datasets of the 
National Survey of Student Engagement (NSSE). Ashley Finley and Tia McNair’s study asks a 
new set of questions related to “impact,” employing an inquiry-based model to probe for answers 
about the cumulative effects of HIPs. Written for campus practitioners, the book makes a unique 
contribution by developing a methodology to support purposeful and intentional study as well as 
equitable implementation of high-impact practices on the ground. Perhaps the most valuable 
findings of the book concern equity—the “equity effects” that appear in students’ reports of 
their learning as their success is boosted by HIPs; the equity-minded perspective that educators 
can nurture; the principles of inclusive excellence that can guide colleges and universities in 
providing a liberal education that offers not only equitable access to HIPs, but also equitable 
achievement of outcomes. 


Working with NSSE data from thirty-eight campuses in three states—California, Oregon, and 
Wisconsin—Finley and McNair advance our collective understanding of what campuses can learn 
from such data by looking specifically at underserved populations among survey respondents. They 
also open a new line of qualitative inquiry by introducing student voices into the HIPs discussion, 
enriching their quantitative conclusions by interpreting results of focus groups conducted with 
underserved students on nine campuses, three in each of the three states. The inquiry-based method 


' George Kuh, High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter (Washington, DC: 
Association of American Colleges and Universities, 2008). 
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and model recommended by Finley and McNair in this mixed-methods study brings welcome 
clarity to aspects of the debate over HIPs that have simply never been studied well. 


In 2008, when High-Impact Educational Practices was published, high-impact practices were by no 
means new. Learning communities had been active since the late 1970s; undergraduate research and 
internships had existed even longer. Kuh’s book caught on because it offered evidence and a 
compelling argument—using indirect assessment and drawing connections based on NSSE and 
other data to make the case for practices many educators had long admired. Kuh, working in 
partnership with AAC&U’s Liberal Education and America’s Promise (or LEAP) initiative, provided 
a new set of definitions, enlarging the number of designated “effective practices” and linking the 
practices to student success in a new way. In retrospect, it seems clear why Kuh’s publication became 
so popular. In the first decade of the twenty-first century, the learning outcomes movement was 
gaining traction on campuses. In 2005, AAC&U had launched LEAP, its centennial campaign. By 
2008, the Essential Learning Outcomes advanced by LEAP were gathering consensus. When 
AACW&U issued High-Impact Educational Practices as a signature LEAP publication, the campaign was 
moving. AAC&U’s membership wanted to address learning outcomes, engaged learning practices, 
and assessment together. Many educators also observed the changing demographics of college 
students and began to wonder how to bring high-impact learning experiences to the new majority. 
As pressure built for productivity, educators began to seek ways to help more students not just 
complete more degrees, but achieve at higher levels. In this environment, the ideas in Kuh’s 
monograph spread rapidly. 


The most powerful recommendation Kuh made in 2008 followed from a single conclusion: 
high-impact practices have a pronounced effect on the experiences of underserved students. 
Using NSSE data, Kuh was able to show generally positive relationships between high-impact or 
engaged experiences and different measures of student learning and achievement, such as 
self-reported gains, grade point averages, and retention. But in several cases, these effects were 
more pronounced for students in identified groups: African American, Latino/a, and students 
with relatively low ACT scores. For example, Kuh was able to show that when they participated 
in high-impact practices, students with lower ACT scores experienced greater boosts to their 
grade point averages than their peers with higher ACT scores. These results pointed to benefits 
from HIPs, not only in the aggregate for all students, but also and particularly for students from 
groups historically underrepresented in higher education—those traditionally least likely to have 
the opportunity to engage in deep learning. Every student, Kuh recommended, ought to 
experience at least two high-impact practices, one in the first two years, the other near 
completing the baccalaureate. Ideally, every student ought to have a high-impact experience 
every year.” 


Since 2008, AAC&U has led the higher education community in promoting access to high-impact 
practices for all students and in pursuing answers to questions about the effect of HIPs on student 
learning outcomes, exploring and advancing our collective progress through these practices toward 
student success. A series of projects and publications ensued. The first multistate project to address 
HIPs was called Give Students a Compass: A Tri-State Partnership for College Learning, General Education, 
and Underserved Student Success (2008-11). With funding from Carnegie Corporation, this project 
aimed at the systemic reform of general education. It rapidly became a large-scale collaborative to 
advance high-impact practices and engaged learning. The Compass project evolved a strategic set of 


2 Kuh, High-Impact Practices, 17-20. 
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recommendations and designs for liberal learning outcomes and inclusive excellence that continue to 
inform AAC&U’s LEAP States Initiative. 


In 2010, AAC&U released Jayne Brownell and Lynn Swaner’s Five High-Impact Practices,? an overview 
of research on five practices that had attracted large numbers of studies. The literature review 
provided persuasive evidence that HIPs do result in higher levels of learning on key learning 
outcomes. But the authors also concluded that we do not know much at all about the learning 
experiences of underserved students. In 2012, AAC&U published Ashley Finley’s Making Progress?,* a 
sober and yet hopeful look at what we can now collectively say about engaged and high-impact 
learning. Truth be told, it is not yet enough. We are still talking in the future conditional: “The 
growing attention to high-impact practices will likely produce richer and deeper sources of evidence 
regarding the efficacy of these practices for all students and, hopefully, for students from underserved 
groups in particular.”° Assessing Underserved Students’ Engagement in High-Impact Practices will help us 
move toward that goal. 


The Assessing Equity in High-Impact Practices Toolkit included in this publication will further 
assist campus practitioners in attending to issues of equity—that is, in becoming “equity-minded” in 
the ways the Center for Urban Education at the University of Southern California’s work 
highlights—as they offer high-impact practices to students. AAC&U is grateful to the Center for 
Urban Education, and particularly to authors Estela Mara Bensimon, Alicia C. Dowd, and Debbie 
Hanson, for sharing this valuable resource with readers of Assessing Underserved Students’ Engagement 
in High-Impact Practices. 


As for the students themselves—underserved students—what do they know and say about their 
learning? This book offers recommendations in the students’ own heartfelt and frank words, 
enlivened by their idiom and their felt experience. They emphasize, of course, the quality of their 
experiences far more than they do the programmatic title or category of the high-impact practice. 
They are absolutely certain that they want real-world applications for learning, and they seek caring 
relationships with adults. They have much good advice to offer. 


Not the least of this book’s effects will be the confidence it inspires among educators to do high- 
impact work on their own campuses, in culturally sensitive and contextually smart ways. We warmly 
hope that Assessing Underserved Students’ Engagement in High-Inpact Practices will prompt educators to 
join in partnership with students to figure out what works. The insights garnered from listening to 
students as they imagine their working lives ahead, as they reflect on the purpose of their learning 
and the experiences that allow them to participate fully as partners in their own success, can give all 
educators many reasons for hope. 


Carol Geary Schneider Susan Albertine 

President Vice President for Diversity, Equity, 
Association of American Colleges and Student Success 

and Universities Association of American Colleges 


and Universities 


> Jayne E. Brownell and Lynn E. Swaner, Five High-Impact Practices: Research on Learning Outcomes, Completion, and Quality 
(Washington, DC: Association of American Colleges and Universities, 2010). 


* Ashley Finley, Making Progress? What We Know about the Achievement of Liberal Education Learning Outcomes (Washington, DC: 
Association of American Colleges and Universities, 2012). 


Finley, Making Progress?, 27. 
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INTRODUCTION 


Exploring the Connection 
between High-Impact Practices 


and Student Success 
=x* * w&e we ke Kh jKrmhUcrY 7K Ke Ke Ke hu Kh hUK 


While educators are scrambling to increase retention and graduation rates to meet the goals of 
the completion agenda, there is increasing evidence about what works to improve students’ 
learning and success. Much is known about the advantages of providing engaged learning 
experiences (often referred to as “high-impact practices”) for students. A cursory scan of campus 
websites will provide at least one, if not multiple, highlights of these practices: students giving 
back to the community, students engaging with faculty, students working collaboratively, 
students engaging in field research, students studying abroad. But when it comes to student 
success and learning, what exactly is the payoff for this engagement? And for whom? 


The publications High-Impact Educational Practices: What They Are, Who Has Access to Them, and 
Why They Matter and Ensuring Quality & Taking High-Impact Practices to Scale helped establish the 
efficacy of high-impact practices using nationwide data from the National Survey of Student 
Engagement (NSSE).' Though such practices have long existed on campuses and are familiar to 
scholars and practitioners alike, with High-Impact Educational Practices, author George Kuh 
provided at least two important insights. First, he offered evidence suggesting that by treating 
these practices as a set of effective tools rather than as discrete experiences, faculty, 
administrators, and other campus professionals could begin to conceptualize the collective 
impact these practices have on indicators of student success and learning. Second, he unveiled 
the need to consider the relationship between participation in high-impact practices and 
underserved student success. 


In recent years, the nationwide postsecondary completion rate has largely stagnated. Between 
1998 and 2008, the average six-year college completion rate in the United States increased by 
only 3 percentage points, to 55.9 percent.* Moreover, graduation rates vary significantly by 
race and socioeconomic background, and gaps between students from groups that have been 
historically underserved by higher education and their peers from traditionally advantaged 
groups are further determined by institutional selectivity. For example, the graduation gap 


' George D. Kuh, High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter (Washington, DC: 
Association of American Colleges and Universities, 2008); George D. Kuh and Ken O’Donnell, Ensuring Quality & Taking 
High-Impact Practices to Scale (Washington, DC: Association of American Colleges and Universities, 2013). 


Nv 


National Center for Higher Education Management Systems Information Center, “Graduation Rates,” accessed January 19, 
2011, http://www.higheredinfo.org/dbrowser/?level=nation&mode=graph&state=0&submeasure=27. Integrated 
Postsecondary Education Data System (IPEDS) graduation rates are based on a defined cohort of first-time, full-time 
students who have not transferred from another institution. 
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favoring white over Hispanic students is 12 percentage points overall, but it expands to 21 
percentage points at highly selective research universities.? 


High-Impact Educational Practices provided insight into the role of high-impact practices in 
ameliorating these gaps. Specifically, it showed compellingly that while both white students and 
black and Hispanic students who participate in “educationally purposeful activities” (a proxy for 
high-impact practices or engaged learning experiences) show improvements, black students’ gains 
in first-to-second-year retention rates and Hispanic students’ gains in first-year grade point 
averages (GPAs) are greater than those of white students. These findings, referred to in that 
publication as a “compensatory effect,” suggest the possibility that high-impact practices, while 
good for all students, might be particularly beneficial for underserved students.* A subsequent 
literature review examining the research on a select group of high-impact practices (first-year 
seminars, learning communities, service learning, undergraduate research, and capstone 
experiences) provided additional support for this hypothesis.* 


As interest in promoting high-impact practices has grown—along with the need to foster and 
sustain student access and success—three critical gaps in the existing research have emerged. First, 
there is scant evidence about the relationship between underserved students’ learning and their 
engagement in high-impact practices. Second, while High-Impact Educational Practices showed that 
Hispanic and black students who engage in high-impact practices demonstrate greater gains in 
their first-year GPAs and a higher probability of first-to-second-year retention, respectively, than 
their white peers, little is known about whether engagement in these practices differentially 
affects learning outcomes for these and other traditionally underserved students. Finally, what is 
known about the nature of student experiences with high-impact practices comes almost entirely 
from surveys, which often fail to reflect the rich detail that is present when students articulate, in 
their own voices, what these experiences mean to them in the context of their lives, their 
learning, and their hopes for obtaining a degree. 


With generous funding from the TG Philanthropy Program,° the mixed-methods study presented 
in this publication aims to fill the aforementioned gaps and advance current understanding of the 
relationship between high-impact practices and underserved students’ success and learning. Part I 
takes a quantitative approach, analyzing NSSE data from a selected group of public universities in 
order to examine the relationship between cumulative participation in high-impact practices and 
students’ perceptions of their own learning, for underserved students as compared to their 
traditionally advantaged peers. Part II complements this quantitative research with a qualitative 
analysis focused on student experiences articulated in students’ own words. Drawing from fifteen 
focus groups conducted on nine campuses with nearly one hundred underserved students 


Laura Horn and C. Dennis Carroll, Placing College Graduation Rates in Context: How 4- Year College Graduation Rates Vary with 
Selectivity and the Size of Low-Income Enrollment (Washington, DC: US Department of Education, 2006), ix, http://nces.ed.gov/ 
pubs2007/2007161-pdf. 


Kuh, High-Impact Educational Practices, 18-19. Note that Pascarella and Terenzini have also identified compensatory effects in 
their work. Ernest T. Pascarella and Patrick T. Terenzini, How College Affects Students (Volume 2): A Third Decade of Research (San 
Francisco, CA: Jossey-Bass, 2005). 


Jayne E. Brownell and Lynn E. Swaner, Five High-Impact Practices: Research on Learning Outcomes, Completion, and Quality 
(Washington, DC: Association of American Colleges and Universities, 2010). 


“Created by the Texas Legislature in 1979, TG is a public, nonprofit corporation that promotes educational access and success so 
that students can realize their college and career dreams. TG offers resources to help students and families plan and prepare for 
college, learn the basics of money management, and repay their federal student loans. In addition, TG administers Federal Family 
Education Loan Program (FFELP) loans made before July 1, 2010, on behalf of the U.S. Department of Education.” TG, 
“Corporate Overview,” accessed April 25, 2013, http://www.tgslc.org/abouttg/overview/. 
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(defined here as underrepresented minority, first-generation, transfer, and low-income students), 
it further explicates how these students connect their learning to their engagement in high- 
impact practices.’ 


In combining quantitative and qualitative approaches, our goal is not only to increase higher 
education’s understanding of the topic in question, but also to provide an inquiry-based 
methodological model for campus practitioners to use as they pursue this work on their own. We 
thus frame each section of the report with guiding questions to help campus practitioners identify 
the evidence they would need to address the challenges of disaggregating data, gathering 
meaningful evidence, and using that evidence to generate more targeted questions—and, hopefully, 
more effective plans for improvement. In the report’s conclusion, we provide a set of 
recommendations for advancing this work on campuses. We see these recommendations as 
launchpads for the next phase of an ongoing exploration of high-impact practices and student 
success and learning. Though we hope that campuses can easily replicate our methodology, we also 
hope practitioners will note the limitations of our research and ultimately improve on our efforts. 


To encourage campus practitioners to engage in a more in-depth analysis of high-impact practices, 
we invited our colleagues from the Center for Urban Education (CUE) at the University of 
Southern California to develop a toolkit for assessing individual high-impact practices based on the 
Equity Scorecard™.* The toolkit, included in appendix A, provides a framework campus leaders can 
use to examine individual high-impact practices from an ““equity-minded’” perspective based on 
who participates in these practices, who has access to them, and what impact participation has on 
identified outcomes. This toolkit helps practitioners examine institutional approaches so that all 
students can benefit from high-impact practices. 


We hope that educators will find all the approaches included in this publication useful as they strive 
to deepen their understanding of high-impact practices on their campuses and to promote student 
success and engagement in the learning experience. 


? We include transfer students as an underserved group due to the expectation that institutions would concentrate high-impact 


practices within the first one or two years, limiting transfer students’ access to such experiences. 


As described on the CUE website, “The Equity Scorecard™ is both a process and a data tool. As a process, it combines a 
theoretical framework with practical strategies to initiate institutional change that will lead to equitable outcomes for students of 
color. What’s unique about it is the engagement of individuals from different departments and divisions as a team in investigating 
campus data, practices and policies.” Center for Urban Education, “The Equity Scorecard,” accessed March 14, 2013, 
http://cue.usc.edu/our_tools/the_equity_scorecard.html. 


Estela Mara Bensimon, “The Underestimated Significance of Practitioner Knowledge in the Scholarship on Student Success,” 
Review of Higher Education 30, no. 4 (2007): 441-69. Equity-mindedness is a concept that locates responsibility for the 
educational success of students of color in institutional practices rather than in student deficits. 
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PART | 


Developing an Inquiry-Based Model 


A Quantitative Approach to Assessing 
Underserved Students Engagement 
in High-Impact Practices 


kK Ke we we we Ke Ke Ke Ke Ke KK kK hum 


In the quantitative portion of this study, we aim to expand what is known about high-impact practices 
by examining how engagement in these practices affects underserved students’ own perceptions of their 
learning (1) relative to students from the same traditionally disadvantaged groups who do not engage in 
these practices, and (2) relative to more traditionally advantaged students who engage at similar levels. 


Several guiding questions informed our analysis: 


1. How do rates of participation in high-impact practices differ among students from various 
underserved and traditionally advantaged groups? 


2. Across different student groups, how does participation in specific high-impact practices 
and in various numbers of these practices affect students’ perceptions of their learning? 


3. Within particular underserved groups, what is the effect of participation in multiple 
high-impact practices on students’ perceptions of their own learning? 


4. How does the relationship between participation in high-impact practices and students’ 
perceptions of their own learning compare between underserved students and their 
traditionally advantaged peers? 


Defining Participation and Perceptions of Learning: Notes on Methodology 

For this study, we identified six high-impact practices: (1) learning communities, (2) service- 
learning courses, (3) study abroad experiences, (4) internships, (5) capstone courses or culminating 
senior experiences, and (6) research with a faculty member. Because previous research has shown 
the efficacy of specific high-impact practices in improving various outcomes,' we primarily focused 
our analysis on the relationship between students’ perceptions of their learning and their cumulative 
participation in multiple high-impact practices. 


To measure students’ combined participation across these six practices, we created a new scale 
drawing from National Survey of Student Engagement (NSSE) data. The scale ranged from zero 


' See, for example, Kuh, High-Impact Educational Practices (see intro., n. 1); Brownell and Swaner, Five High-Impact Practices (see intro., n. 5); 
Kuh and O’Donnell, Ensuring Quality (see intro., n. 1). 
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(for students who responded that they had not participated in any of the identified high-impact 
practices) to six (for students who reported participating in all six practices).? In the case of service 
learning, NSSE asks students to respond to the question, “In your experience at your institution 
during the current school year, about how often have you participated in a community-based 
project (e.g., service-learning) as part of a regular course?” If a student responded “sometimes,” 
“often,” or “very often,” we counted him or her as having participated in the practice. For the other 
five high-impact practices, NSSE asks students, “Which of the following have you done or do you 
plan to do before you graduate from your institution?” In those cases, we counted participation 
according to whether a student responded that he or she “had done” a particular practice. 


We compared students’ participation in these six practices to students’ perceptions of their own 
learning, as reflected through four measures: (1) self-reported engagement in activities that NSSE 


researchers associated with “deep approaches to learning,’ 


(2) self-reported gains in practical 
competence, (3) self-reported gains in general education, and (4) self-reported gains in personal 
and social development.* These measures represent a composite of students’ responses to a series 
of items about their educational experiences and perceived learning gains. NSSE researchers have 
standardized these composite responses to a one-hundred-point scale, with higher numbers 
indicating that a student reported having experienced more significant gains in learning or 
having engaged in more experiences connected to deep learning. (See appendix B for a full list 


of the items used to construct the four deep learning and gains scales.)° 


Our data set included NSSE data from 25,336 students at thirty-eight institutions across the state 
higher education systems in California, Oregon, and Wisconsin.° We selected these state systems 
because of their prior engagement in the Association of American Colleges and Universities’ 
(AAC&U’s) Liberal Education and America’s Promise (LEAP) States Initiative, a systemic change 
effort designed to build platforms for campus action and frameworks to advance essential learning 
outcomes in general education and across institutional operations. These “LEAP states” have 
adopted AAC&U’s Essential Learning Outcomes system-wide as a guiding framework for 


N 


Notably, this scale could not account for a student having participated in multiple high-impact practices of the same type 
(e.g., multiple undergraduate research experiences). Although student responses related to how often they had participated in service 


20 ee 


learning (i.e., “never,” “sometimes,” “often,” “very often’’) do indicate participation in multiple service-learning experiences, to preserve 


continuity within the scale, we treated the categories of “sometimes,” “often,” and “very often” as a single indicator of participation. 


Kuh, High-Impact Educational Practices, 14. 


This analysis builds on Kuh’s earlier work on the “compensatory effect”? of high-impact practices, as reflected in Kuh, High-Impact 
Educational Practices, 17-19. As described in the introduction to this publication, Kuh reported that some underserved students who 
engaged in high-impact practices experienced higher gains than their traditionally advantaged peers in regard to first-year GPA or 
first-to-second-year retention, compensating for performance gaps in these areas. The definition of “high-impact practices” used here 
differs from that used in Kuh’s work. In High-Impact Educational Practices, Kuh assessed the “compensatory effect” of high-impact 
practices for underserved students using a scale of “educationally purposeful activities,’ composed of nineteen items encompassing a 
range of activities including but not limited to specific high-impact practices. While the “educationally purposeful activities” scale thus 
included items asking students whether they had “participated in community service as part of a course,” most items were not 
connected with specific high-impact practices and instead reflected more broadly defined “educationally purposeful” activities 

(e.g., “worked with other students on projects during class” and “tutored or taught other students”). Thus, while Kuh’s findings were 
based on a broad articulation of activities defined as “educationally purposeful,” this study focuses on the cumulative effect of 
experiences specifically labeled as “high-impact” (learning communities, service-learning courses, study abroad experiences, internships, 
capstone courses or culminating senior experiences, and research with a faculty member). This clarification is not to take issue with the 


“educationally purposeful activities” scale or its usefulness. It is intended only to distinguish how this analysis varies from Kuh’s work. 
For additional information, visit the NSSE website at http://nsse.iub.edu/_/?cid=368. 


Each campus provided one year of data collected between 2006 and 2008. With the consent of campus representatives, NSSE 
researchers pooled the institutional data into a single dataset, which they submitted to AAC&U for analysis. No institutional or 
student identifiers were included in the final dataset. 
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institutional-level curricular reform and assessment. However, the degree to which each institution 
has officially adopted these outcomes varies extensively, and we do not assume that students on 
these campuses are particularly engaged in or familiar with high-impact educational practices. 


Students included in the sample possessed a diverse range of personal characteristics (see table 1.1).’ 
For the purposes of this study, we focused our analysis on three different categories of 
underserved students—first-generation, transfer, and underrepresented racial or ethnic minority 


traditionally advantaged students.* In conducting Table 1.1 
our analysis, we took into account and controlled Sample Characteristics 


students—and on comparison groups of 


for differences in aoe gender, age, class level, leper Seat Percentage of Sample 
and (where appropriate) race or ethnicity, transfer (n = 25,336) 
status, and first-generation status.” Accounting for Gender 


these additional, or confounding, factors helps Female 63% 


eliminate alternative explanations for differences 


in survey responses and bolsters evidence for the ave 
unique effect high-impact practices have on sign) 39% 
students’ perceptions of their learning. 20-23 | 37% 
24+ | 24% 

Class Rank 


Results: Addressing Guiding Questions for 
Underserved Student Success and 
Establishing Evidence for Improvement 
We used the series of questions that opened this 


First-Year | 37% 


Sophomore | 5% 


Junior | 5% 


Senior | 51% 


Unclassified | 2% 


chapter to structure our analysis of how participation 


in high-impact practices affects students’ perceptions 


of their engagement in deep learning and of their Race or Ethnicity 
learning gains. We encourage campus practitioners White | 59% 
to apply these questions when examining their own African American | 3% 


data to help guide the intentionality and 
thoughtfulness of their own local efforts. 


Hispanic | 13% 


Asian American | 12% 


Other | 6% 
How do rates of participation in high-impact Declined Response | 7% 
practices differ among students from various Transfer Status 


underserved and traditionally advantaged groups? 
Transfer | 33% 


First-Generation Status 


First-Generation | 51% 


On average, students in the sample engaged in 


between one and two (1.3) high-impact practices. 


However, participation varied among different 


’ Although NSSE targets first-year and senior students, the sample also included students who self-identified as sophomores or juniors. 
The analysis controlled for the potential impact of class level on outcome measures. Due to the small percentage of Native American 
students in the sample, this racial category was combined with students who identified their race as “multiracial” or “other.” 


* We used an ordinary least squares regression analysis to obtain these results. Differences are significant at p<.001. 


’ Tn analyzing particular underserved student groups (e.g., transfer students), we accounted for the fact that students might 
simultaneously belong to one or more other underserved group (e.g., underrepresented racial or ethnic minority groups). 
For example, when examining the effects of engagement in high-impact practices for first-generation and non-first-generation 


students, we controlled for transfer status and race. 
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groups of students. Table 1.2 provides a breakdown of the average number of high-impact 


practices in which students engaged, by underserved and traditionally advantaged groups. (See 


appendix C for a breakdown of the total number of students who participated in cumulative 


high-impact practices by class status, gender, and underserved category.) 


Table 1.2 


Average Number of High-Impact Practices by Underserved and Traditionally 


Advantaged Group 


Average Number of High-Impact Practices 


Average Number of High-Impact Practices 


(Underserved) (Traditionally Advantaged) 
First-Generation | 1.24 Not First-Generation | 1.45 
Transfer | 1.53*** Non-Transfer | 1.25 
African American | 1.29 White | 1.38 
Hispanic | 1.27* 
Asian American | 1.22* 


EK 


*=p<.o5, ““=p<.o1, ***=p<.oo1 

As shown in table 1.2, transfer students in the sample engaged in significantly more high-impact 
practices than non-transfer students (1.53 practices versus 1.25 practices, respectively). In 
contrast, first-generation students engaged in significantly fewer high-impact practices than 
students who were not first generation. While white students did, on average, participate in 
more high-impact practices (1.38) than students from other racial or ethnic categories, the 
difference in participation rates between white students and African American students was not 
statistically significant.'” White students did, however, engage in significantly more high-impact 
practices than Asian American and Hispanic students.'! 


These results underscore the need to carefully examine who is participating in high-impact 
practices. When evaluating the effects of high-impact practices and targeting related 
programmatic improvements, it is essential to gather evidence of the degree to which particular 
groups of students are or are not participating in these practices.'” 


Across different student groups, how does participation in specific high-impact practices and in 
various numbers of these practices affect students’ perceptions of their learning? 


To fully understand the effects of high-impact practices, we sought to assess baseline or benchmark levels 
of self-reported learning. Baseline or comparison data can be collected either prior to starting a particular 
intervention or from a comparison group that has not experienced the intervention. For this study, we 
used as our baseline the perceptions of students who had not participated in any high-impact practices. 


The participation of white students in high-impact practices also did not vary significantly from the participation of students 
whose racial or ethnic category was coded as “other” or students who declined to identify their race or ethnicity. Students in 
these categories reported participating in an average of 1.4 high-impact practices. 


With the exception of transfer students, whom NSSE found to have lower participation in high-impact practices than 
non-transfer students, these findings echo the participation data published in Kuh, High-Impact Educational Practices, 16. The 
difference between these results and those reported by NSSE regarding transfer students are likely due to the inclusion of 
students from all class years in this analysis. NSSE analyzes data for first-year and senior students only and reports those 
findings separately. 


See focus group findings in Part II for student perspectives on barriers to participation in high-impact practices. 
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Using these baseline data, we assessed the degree to which participation in specific high-impact 
practices affects students’ perceptions of their own learning. Looking at the effect of participating 
in each of the six high-impact practices, we found that students who participated in any single 
high-impact practice perceived their learning significantly more positively than students who did 
not participate in that same practice (see figure 1.1). 


Students who participated either in a service-learning experience or in student/faculty research 
reported levels of engagement in deep learning and perceived gains that were an average of 8.1 
points higher on the standardized scale than those of students who did not participate in these 
practices. Similarly, students who participated in either a learning community or a senior 
capstone experience reported levels of engagement in deep learning and perceived learning gains 
that were an average of 7.7 points and 6.1 points higher, respectively, than those of students who 
did not participate in these practices. Students who participated in an internship or study abroad 
experience also reported higher average levels of engagement in deep learning and perceived 
gains (5.2 points and 4.3 points on the scale, respectively) than students who did not participate, 
although the differences were not as large as for other high-impact practices. 


Figure 1.1 


Average Boost to Students’ Perceptions of Their Deep Learning and Gains 
by Participation in Specific High-Impact Practices 


Learning Service Study Internship Student/Faculty Senior 
Community Learning Abroad Research Capstone 
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High-Impact Practice 


Next, we examined the effect of participation in multiple high-impact practices on students’ 
perceptions of their own learning. Figures 1.2 and 1.3 show that there is a measurable, 
significant, and positive relationship between students’ cumulative participation in multiple 
high-impact practices, on the one hand, and their perceived engagement in deep learning and 
their perceived gains in learning, on the other. 


'S Mean differences between groups across all high-impact practices examined were significant at p<.001. 
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Figure 1.2 
Deep Learning Experiences by Cumulative Participation in High-Impact Practices (HIPs) 
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Deep Learning 


Figure 1.3 
Self-Reported Gains by Cumulative Participation in High-Impact Practices (HIPs) 


Standardized Scale (0-100) 


Gains in Gains in Gains in 
General Education Practical Competence Personal and Social Development 
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Figure 1.2 shows that students who participated in greater numbers of high-impact practices reported 
consistently higher levels of perceived engagement in deep approaches to learning. As demonstrated in 
figure 1.3, the same was true with regard to students’ perceptions of their gains in general education, 
practical competence, and personal and social development. It is notable that students’ perceived gains 
with regard to personal and social development, regardless of whether students participated in high- 
impact practices or how many practices they participated in, were consistently lower than their 
perceived gains in other areas—a trend that has been observed in previous studies." 


Given these findings across groups, we wanted to know how perceptions of their learning vary among 
students from underserved populations relative to students from traditionally advantaged groups. The 
next two sections of this part address the effects of participation in high-impact practices on students’ 
perceptions of their own learning, first for students within particular underserved groups, and 
subsequently for underserved students overall relative to students from traditionally advantaged groups. 


Within particular underserved groups, what is the effect of participation in multiple 
high-impact practices on students’ perceptions of their own learning? 

Within-group comparisons of the relationship between participation in multiple high-impact 
practices and perceptions of learning suggest overwhelmingly that students within specific 
underserved categories benefit significantly from engaging in these activities. Figure 1.4 shows the 
average boost across all four perceived learning measures (engagement in deep learning, gains in 
general education, gains in practical competence, and gains in personal and social development) 
when first-generation and transfer students participated in multiple high-impact practices, as 
compared with students in these same groups who did not participate in these practices. 


Specifically, first-generation students who participated in one or two high-impact practices 
reported levels of engagement in deep approaches to learning and perceived gains that were, on 
average, 11 percent higher on the standardized scale than those of first-generation students who 
did not participate in a high-impact experience. When first-generation students participated in 
three or four high-impact practices, their levels of engagement in deep learning approaches and 
their perceived gains were, on average, 24 percent higher than those of first-generation students 
who did not participate in a high-impact experience. Finally, if these students participated in five 
or six high-impact practices, their levels of reported engagement in deep learning approaches and 
their perceived gains were, on average, 35 percent higher than those of their first-generation 
peers who did not participate. Similar results were found for transfer students. 


See Ashley Finley, Making Progress? What We Know about the Achievement of Liberal Education Learning Outcomes (Washington, 
DC: Association of American Colleges and Universities, 2012), 9-11. 
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Figure 1.4. 

Within-Group Comparisons for First-Generation and Transfer Students: 
Average Boost in Deep Learning Experiences and Self-Reported Gains with 
Multiple High-Impact Practices (HIPs) vs. No Participation 
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Like first-generation and transfer students, students in underrepresented racial or ethnic minority 
groups also benefitted significantly from engagement in multiple high-impact practices. As shown 
in figure 1.5, within specific racial or ethnic groups, students’ levels of reported engagement in 
deep approaches to learning and their perceived learning gains were significantly higher for those 
who engaged in multiple high-impact practices than for those who did not engage in these 
practices.'° For Hispanic students, for example, levels of reported engagement in deep learning 
approaches and in perceived gains were 10 percent higher on average with one or two high- 
impact practices, 17 percent higher with three or four practices, and 26 percent higher with five 
or six practices, compared to Hispanic students who did not participate in any high-impact 
practices. Similarly, when African American students participated in multiple high-impact 
practices, their perceived engagement in deep learning and their learning gains were between 11 
and 27 percent higher (depending on the level of engagement) than that of African American 
students with zero participation in these practices. Asian American and white students exhibited 
similar boosts in their perceived deep learning and in their learning gains after engaging in 
multiple high-impact practices, particularly when they had engaged in five or six practices. 


' Most of these gains were significant at p<.001. Summary results are available from the authors upon request. 
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Figure 1.5 

Within-Group Comparisons by Racial or Ethnic Category: Average Boost in 
Deep Approaches to Learning and Self-Reported Gains in Learning with 
Multiple High-Impact Practices (HIPs) vs. No Participation’® 
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In sum, the effects of engagement in multiple high-impact practices, both across and within different 
groups of students, provide strong evidence for the need to ensure that all students encounter several 
high-impact experiences during college. Our analysis suggests that curricula developed with an eye 
toward pervasiveness of high-impact practices across years are likely to provide a positive and 
discernible “boost” to students’ perceptions of their learning, regardless of their background.'” 


How does the relationship between participation in high-impact practices and students’ perceptions of 
their own learning compare between underserved students and their traditionally advantaged peers? 


The preceding analysis indicates that students who participate in multiple high-impact practices perceive 
their learning more positively than students in the same group who do not participate in high-impact 
activities. To add nuance to this finding, we examined the degree to which engagement in multiple 
high-impact practices has a comparable effect on perceptions of learning between different groups of 
students, particularly those that are underserved relative to students from traditionally advantaged groups. 


Results for students whose racial category was classified as “other,” along with those for students who declined to identify their 
racial categories, are not included in the graph. Results for those categories are available from the authors on request. Percentage 
differences were significant using a p-value of at least p<0.05, with two exceptions: the average percentage difference for Hispanic 
students with five or six high-impact practices relative to Hispanic students who did not participate in any high-impact practices 
(significant at p<0.01), and the average percentage difference for African American students with five or six high-impact practices 
relative to African American students who did not participate in any high-impact practices (not statistically significant). 


g 


George Kuh makes a similar recommendation in Kuh, High-Impact Educational Practices, 19. 
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As in the preceding within-group analysis, the comparison of data between different groups of 
students suggests that, overall, engagement in high-impact practices positively affects students’ 
perceptions of their learning. Tables 1.3 and 1.4 compare self-reported engagement in deep 
learning and self-reported learning gains at different levels of participation in high-impact practices, 
for first-generation students compared to non-first-generation students and for transfer students 
compared to non-transfer students. 


Table 1.3 
Deep Learning Experiences and Self-Reported Gains by First-Generation Status 
and Cumulative Participation in High-Impact Practices (HIPs) 


First-Generation Students | 54.27 61.00 68.13 74.83 


Not First-Generation Students | 54.46 61.14 67.56 74.70 
(Difference) | (-0.19) (-0.14) (0.57) (0.13) 

First-Generation Students 67.29 71.54 77.30 80.29 

Not First-Generation Students | 63.67 68.97 75.01 79.62 
(Difference) | (3.62) (2.57) (2.29) (0.67) 

First-Generation Students | 59.91 66.85 73-71 78.46 

Not First-Generation Students 57.82 65.46 71.91 78.02 

(Difference) | (2.09) (2.39) (1.8) (0.44) 

First-Generation Students | 46.25 51.79 57-60 66.32 

Not First-Generation Students | 44.07 50.00 54.83 65.43 

(Difference) | (2.21) (1.79) (2.77) (0.89) 


Numbers reflect deep learning and gains as calculated according to a one-hundred-point scale. 
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Table 1.4, 


Deep Learning Experiences and Self-Reported Gains by Transfer Status and 
Cumulative Participation in High-Ilmpact Practices (HIPs) 


Deep Learning 
Transfer Students 56.40 62.75 69.21 75.01 
Non-Transfer Students 53.67 60.15 66.93 74.60 
(Difference) | (2.73) (2.60) (2.28) (0.41) 
Gains in General Education 
Transfer Students 64.92 70.24 74.92 78.29 
Non-Transfer Students 65.93 70.33 76.73 80.83 
(Difference) | (-1.01) (-0.09) (-1.81) (-2.54) 
Gains in Practical Competence 
Transfer Students 59.20 66.56 71.89 77.63 
Non-Transfer Students 58.90 65.97 73.22 78.54 
(Difference) | (030) (0.59) (-1.33) (-0.91) 
Gains in Personal and Social Development 
Transfer Students 40.90 47-54 53-97 64.89 
Non-Transfer Students 46.80 52.77 57.38 66.33 
(Difference) | (-5.90) (-5.23) (-3.41) (-1.44) 


Numbers reflect deep learning and gains as calculated according to a one-hundred-point scale. 


As table 1.3 shows, the differences in perceptions of learning between first-generation and non- 
first-generation students tended to be smaller when students engaged in greater numbers of 
high-impact practices. For first-generation and non-first-generation students, for instance, the 
difference in perceived gains in general education became smaller with each successively higher 
level of participation in high-impact practices, shrinking from a difference of 3.62 points with zero 
engagement in high-impact practices to a difference of 0.67 points at five or six high-impact 
practices. Overall, results for first-generation and non-first-generation students suggest that 
disparities in how students perceive their learning are reduced when students engage in greater 
numbers of high-impact practices. A similar trend appears for transfer and non-transfer students in 
their reported deep learning and perceived gains in personal and social development (see table 1.4). 


The relative effects of engagement in high-impact practices were also evident when comparing 
students from racial or ethnic minority groups with each other and with their traditionally 
advantaged white peers. While participation in high-impact practices yielded positive effects on 
students’ perceptions of their learning and reported learning gains across all racial or ethnic groups, 
in some cases student groups that reported the lowest perceived deep learning or gains in learning 
absent high-impact practices demonstrated the greatest boosts in these perceptions when they had 
engaged in multiple high-impact practices. As shown in figure 1.6, African American, Hispanic, and 
white students all demonstrated higher levels of engagement in deep learning approaches after 
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participating in greater numbers of high-impact practices. However, the persistent and substantial 
boost in Asian American students’ perceptions of their learning with increasingly greater levels of 
high-impact participation, relative to these other racial or ethnic groups, is particularly noteworthy. 


Figure 1.6 
Deep Learning Experiences by Race or Ethnicity and Cumulative Participation 
in High-Impact Practices (HIPs) 
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Similarly, figure 1.7 shows an initial boost in perceptions of gains in general education for African 
American and Hispanic students as they engaged in one to four high-impact practices, with this 
effect leveling off or declining slightly at more than four practices. However, for Asian American 
and white students, engagement in greater numbers of high-impact practices had a persistent and 
increasingly positive effect on their perceptions of gains in general education. 
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Figure 1.7 
Self-Reported Gains in General Education by Race or Ethnicity and Cumulative 
Participation in High-Impact Practices (HIPs) 
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Comparisons of perceived gains in practical competence and personal and social development (figs. 
1.8 and 1.9) illustrate similar trends. Both figures show general upward trends in perceptions of 
learning in these areas when students across racial groups engaged in greater numbers of high- 
impact practices. Of particular note is the boost in Asian American students’ perceptions of their 
gains in these areas when they engaged in five or six high-impact practices. In contrast, African 
American students’ perceptions of their gains in practical competence and personal and social 
development showed only slight improvement with engagement in five or six practices, particularly 
as compared to other racial or ethnic groups. 
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Figure 1.8 
Self-Reported Gains in Practical Competence by Race or Ethnicity and Cumulative 
Participation in High-Impact Practices (HIPs) 
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Figure 1.9 
Self-Reported Gains in Personal and Social Development by Race or Ethnicity and 
Cumulative Participation in High-Impact Practices (HIPs) 
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Gains in Personal and Social Development, Standardized Scale (0-100) 
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The preceding tables and figures illustrate how students of different backgrounds—including 
underrepresented minority, transfer, and first-generation students—view and understand their 
learning when they have engaged in various numbers of high-impact practices. These findings 
suggest that different groups of students benefit both similarly and differently from participating 
in multiple high-impact practices. In some cases, differences in perceptions of learning between 
groups of students were smaller at greater levels of engagement. In other cases, groups of 
students who reported lower levels of engagement in deep approaches to learning or fewer 
learning gains absent high-impact participation appeared to benefit substantially from 
involvement in these practices. Their engagement provided a boost in their perceptions of their 
own learning that was even greater than the boost exhibited by comparable peer groups. And in 
nearly every case and with almost every group examined, underserved or not, students perceived 
their deep learning experiences and gains in learning more positively with greater levels of 
engagement in high-impact practices. 


Despite these positive effects across groups, to simply state that “high-impact practices are good for 
everyone” would be to miss the various ways in which different groups of students experience these 
practices and benefit from engaging in them. In order to fully understand the effects of participation 
in high-impact practices for any single group of students, campus practitioners need to understand 
those effects in context with data on other groups. Such comparisons provide a means for 
interpreting the magnitude of the benefits of participation in high-impact practices—for whom, 
relative to what baseline, and compared to whom. 


The evidence that high-impact practices provide distinctive and compelling benefits for multiple 
groups of students, including those who have been traditionally underserved in higher education, 
illustrates what might best be referred to as the “equity effects” of high-impact practices. In the 
context of this analyses, such equity effects vary, but they include smaller gaps in perceived learning 
at higher levels of participation in high-impact practices and larger boosts for particular groups that 
view their learning less positively in the absence of such practices, as described above. However, 
these national data are too far removed from the nuances of campus life and policy making to guide 
adjustments in campus practice or to inform new efforts. It is imperative, therefore, for campus 
practitioners to examine their own equity effects by disaggregating their data and comparing their 
findings. Such analyses can provide the information needed for evidence-based decisions about how 
to improve programs that strive toward, advance, and support equitable outcomes for students across 
all groups, particularly underserved students. 


Discussion 

This analysis lends support to the already strong evidence that high-impact practices matter in 
higher education. It also expands existing knowledge by illuminating how participation in not just 
single but multiple high-impact practices can influence students’ perceptions of their learning. 

In particular, this research provides important insights into how high-impact practices influence 
perceptions of learning among students from underserved groups. 


No research is without limitations. Certain shortcomings of the data and methodology should be 
considered when interpreting and applying the findings. First, because the dataset did not include 
information about socioeconomic status, it was not possible to account for this variable either 
within or across underserved student groups. Additionally, the dataset did not allow us to 
disaggregate the sample of Asian American students into ethnic subgroups. Treating “Asian 
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American” as a homogeneous group fails to take into account the enormous diversity of access and 
advantage within this population. Research that increases our understanding of the degree to which 
participation in high-impact practices affects perceptions of learning among students from 
economically disadvantaged backgrounds and among Asian American subpopulations will be critical 
extensions of this work. 


Second, the data were unable to account for students’ engagement in multiple high-impact practices 
of the same type. When considering the impact of cumulative high-impact practices, it may be helpful 
to examine how repeated engagement in the same type of high-impact practice (e.g., multiple service- 
learning experiences) affects students’ perceptions of their learning. Campus researchers will want to 
consider ways to address this issue as they identify and track high-impact practices. 


Third, surveys almost always carry the risk of selection bias. It is possible that students with 
relatively high levels of engagement may have chosen to complete the survey, thereby skewing 
results in the direction of greater engagement than actually exists. Self-reported data also carry 
certain risks of misperception—in this case, related to the ways in which students perceive certain 
practices and the impact of those practices on their learning. To confront these biases and more 
clearly discern impact over time, campus practitioners should consider conducting “pre- and 
post-tests”—that is, assessments of learning before and after engagement in high-impact activities. 
Campus practitioners should also consider directly assessing student learning by examining 
coursework connected with high-impact practices, thereby gathering vital information about the 
connection between these practices and intended learning outcomes. Practitioners can help 
ameliorate selection bias by collecting data from assignments linked to particular high-impact 
practices. The use of such assignments for assessment also means that students have a vested interest 
in actually demonstrating their learning. 


Ultimately, our findings strongly suggest that students’ views of their learning benefit from 
engagement in multiple high-impact practices, whether students are underserved or traditionally 
advantaged. While student perceptions are an indirect measure of outcomes, these perceptions matter. 
Data on how students view their own learning can provide clues about what works, what doesn’t, and 
how education changes people. In the next section, we share findings from focus groups where 
students’ own voices provided hints about what different learning experiences have meant to them. 
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PART II 


Developing an Inquiry-Based Model 


A Qualitative Approach to Assessing 
Underserved Students Engagement 
in High-Impact Practices 


kK we we we we Ke Ke Ke Ke Ke Ke Kh K hum 


Part I of this publication illustrated a process for using quantitative data to assess the relationship 
between high-impact practices and students’ perceptions of their own learning. This process, with its 
focus on the potential “equity effects” of high-impact practices, is a critical first step in exploring 
student success and developing guiding questions to inform decision making on campuses. However, 
institutions should not rely solely on quantitative data, as the learning process is too complex to be 
reduced to a set of data points. By engaging in continual inquiry into the student experience, campus 
practitioners can begin to disentangle quantitative findings and gather instructive evidence about their 
own campus cultures and educational contexts for different student groups. By striving to understand 
the lived experience of students—and their engagement with their learning environments, in 
particular—educators can gather useful evidence regarding potential areas for improvement related to 
student success. Such inquiry is particularly useful when identifying ways to improve academic success 
for underserved students. 


Several guiding questions informed our analysis: 


1. How do underserved students connect their college learning with future workforce 
preparation? 


2. What learning experiences do underserved students value in developing the skills and 
competencies they view as important to employers? 


3. What factors do underserved students identify as barriers or obstacles to their participation in 
high-impact learning experiences? 


Overview of Focus Group Methodology 


Because student voices are essential when examining students’ perceptions of engaged learning, they 
were a key component of our effort to create an inquiry-based process by which campuses can 
collect useful evidence to inform decision making. During spring 2012, we conducted fifteen focus 
groups with students at nine comprehensive public institutions in states actively involved with the 
Liberal Education and America’s Promise (LEAP) States Initiative, a systemic change effort designed 
to build platforms for campus action and frameworks to advance the Association of American 
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Colleges and Universities’ (AAC&U’s) Essential Learning Outcomes in general education and across 
institutional operations.' We designed these focus groups to explore the educational experiences of 
underserved students, with the aim of addressing the stagnant completion rates and growing 
educational disparities between underserved and traditionally advantaged students described in the 
introduction to this report. 


To assist us in recruiting students for the focus groups, each participating institution submitted a 
list of students classified as traditionally underserved by higher education (i.e., minority, Pell-grant 
eligible, transfer, and/or first generation) along with their e-mail addresses. We randomly selected 
students from each list to receive electronic invitations to participate in a focus group. The invitations 
explained the purpose of the focus groups and asked recipients to complete a recruitment survey and 
to submit their e-mail addresses again if they agreed to participate.2 We randomly selected focus group 
participants from among survey respondents. 


Using data gathered through survey responses, we divided participants into two groups: students who 
had participated in three or more engaged learning practices, and students who had participated in 
two or fewer. Of the ninety-one students who ultimately participated in the focus groups, only 
seventeen self-identified as having participated in two or fewer engaged learning practices. While we 
initially intended to compare responses across two groups of students—those who had and those who 
had not participated in high-impact practices—we were thus unable to identify a comparison group 
of nonparticipants. Forty students self-identified as transfer students, forty-six students identified as 
low-income, and forty-eight students identified as being from the first generation in their families to 
attend college. The focus groups were racially and ethnically diverse, consisting of twenty-eight 
Hispanic students, twenty-four African American students, fourteen multiracial students, eleven Asian 
American students, nine white students, and one Arab American student. 


We conducted focus groups on each of the nine participating campuses in March and April 2012.* 
To moderate the groups, we used a script of questions covering a range of topics, including the 
meaning and purpose of a college education, the skill sets and knowledge employers are seeking in 
recent college graduates, what it means to be engaged in learning, and the types of environments that 
foster engaged learning. (The script used for the focus groups is provided in appendix D.) An outside 
vendor transcribed the focus group recordings, and we analyzed the transcripts to identify recurring 
themes with the help of two research assistants using NVivo software.° 


Focus groups were facilitated by Tia Brown McNair, senior director for student success, and Ashley Finley, senior director of 
assessment and research, both of the Association of American Colleges and Universities. The institutional review board of each 


participating institution granted approval prior to the focus group meetings. 


Nu 


The survey asked respondents to give their informed, voluntary consent to participate in the research project and to respond to 
several demographic questions (regarding academic year, transfer status, highest level of parental education, and racial or ethnic 
categories). The survey also described particular learning experiences—first-year seminars and experiences, common intellectual 
experiences, writing-intensive courses, collaborative assignments and projects, undergraduate research, diversity and global 
learning, service and community-based learning, internships, and capstone courses and projects—and asked respondents to 
identify those experiences in which they had participated during college. During the focus groups, the researchers referred to 
the learning experiences as “engaged learning” instead of “high-impact practices” in order to minimize the use of jargon that 
may be unfamiliar to students. Survey respondents were notified that they would receive a $25 gift card if they were selected for 
and agreed to participate in a focus group. 


Some participants chose not to identify their racial or ethnic categories. 


Focus groups were videotaped and audiotaped. Each participating student signed an informed consent form and gave permission 


for use of the video clips in presentations. 


° NVivo software provides a way for researchers to collect, organize, and analyze qualitative content gathered through interviews, 


focus groups, surveys, and audio recordings. See http://www.gsrinternational.com/products_nvivo.aspx. 
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Although focus group participants represented a wide array of student groups that have been classified 
as underserved in higher education, the small number of participants calls for caution in generalizing 
our findings. Nonetheless, we identified themes and messages that were consistent across campuses, 
and we highlight those areas of commonality here. Our goal was not to make definitive statements or 
to establish causal relationships, but to learn more about students’ lived experiences within their 
learning environments, and especially about their views of engaged learning practices. 


Results: Addressing Guiding Questions to Understand Underserved Student Learning 
in High-Impact Practices 
How do underserved students connect their college learning with future workforce preparation? 


Through its national Liberal Education and America’s Promise (LEAP) initiative, AAC&U advocates 
high-impact practices as a way to help students achieve the Essential Learning Outcomes (ELOs) that 
are part of a framework for student success and workforce preparation (see figure 2.1 on next page). 


AAC&U developed the ELOs after engaging in a multiyear dialogue with hundreds of colleges and 
universities about the goals of student learning; analyzing recommendations and reports from the 
business community; and analyzing accreditation requirements for schools of engineering, business, 
nursing, and teacher education. A host of higher education institutions have adopted or modified 
the ELOs since their original publication in 2007. Still, the question remains: Are students able to 
identify the ELOs as critical components of their preparation for the workforce? 


To address this question, we asked focus group participants, “What do you believe potential 
employers are looking for in college graduates? What are the specific skills that you are learning (or 
hope to learn) in college that are important in the professional world?” Focus group participants did 
not receive a copy of the ELOs prior to convening, so their answers drew on their own personal 
experiences and knowledge. 


Students consistently identified the following skills, capacities, and areas of knowledge as important 
in the workforce: 


> oral and written communication skills 

critical and analytical thinking 

ability to work well with others/teamwork 

leadership skills 

ability to apply knowledge to solve problems 

experience working with people from diverse backgrounds 
strong work ethic and values 

creativity in problem solving 

hands-on experience 


desire for additional knowledge 


Vv Vv VV VV VV WV WV WV 


global knowledge 
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Figure 2.1 
The LEAP Essential Learning Outcomes 


The Essential Learning Outcomes 
xX &*e w&w&e wel KermhUc CY Ke Ke hKe Kh Khu 7K KK Kk ® 


Beginning in school, and continuing at successively higher levels across their college studies, 
students should prepare for twenty-first-century challenges by gaining: 


+ Knowledge of Human Cultures and the Physical and Natural World 


* Through study in the sciences and mathematics, social sciences, humanities, histories, 
languages, and the arts 


Focused by engagement with big questions, both contemporary and enduring 


* Intellectual and Practical Skills, including 


* Inquiry and analysis 

* Critical and creative thinking 

¢ Written and oral communication 
* Quantitative literacy 

* Information literacy 

* Teamwork and problem solving 


Practiced extensively, across the curriculum, in the context of progressively more challenging 
problems, projects, and standards for performance 


+ Personal and Social Responsibility, including 


* Civic knowledge and engagement—local and global 
¢ Intercultural knowledge and competence 

* Ethical reasoning and action 

¢ Foundations and skills for lifelong learning 


Anchored through active involvement with diverse communities and real-world challenges 


+« Integrative and Applied Learning, including 
* Synthesis and advanced accomplishment across general and specialized studies 


Demonstrated through the application of knowledge, skills, and responsibilities to new settings 
and complex problems 


Note: This listing was developed through a multiyear dialogue with hundreds of colleges and universities about needed goals for student 
learning; analysis of a long series of recommendations and reports from the business community; and analysis of the accreditation require- 
ments for engineering, business, nursing, and teacher education. The findings are documented in previous publications of the Associa- 
tion of American Colleges and Universities: College Learning for the New Global Century (2007) and The LEAP Vision for Learning (2011). 
For more information, see www.aacu.org/leap. 


LEAP 
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Among the skills, capacities, and areas of knowledge they named, participants mentioned 
communication skills as a primary competency that is necessary for employability: 


“T feel like the number one thing that they [employers] look at is . . . your verbal and written communication 
skills, because I’ve been through a few interviews, and that’s . . . the one thing that I see, and that they 
[employers] tell me.”° 


“To be able to communicate what you’re doing in a job or how you do it is very important, and I think . . . 
[that someone who is able to do that is] the ideal employee.” 


“T feel like the people who are the most successful are the ones who can communicate their vision.” 


Along with communication skills, students also identified the ability to apply knowledge or to work as 
a team as essential for workforce preparation. For example, one student commented, “I think what 
[employers] want would be just listening in general and having that experience to apply what you 
learned previously . . . to the job. Listening is always a crucial element . . . [in] any job and [so is] being 
able to communicate . . . with other people and being able to work with people.” Additionally, students 
highlighted critical thinking and problem solving as important skills for the job market: 


“{T]hinking critically is . . . going to help you with everything else in life; once you learn that, you’re good 
to go anywhere.” 


[T]he biggest thing is critical thinking: being able to analyze . . . societal issues, but then on top of that, . . . 
[being] able to problem solve.” 


Students often said that hands-on experience is necessary for employability. According to one student 
who was participating in a practicum, “I think employers look for experience. . . . [A] lot of people . . . 
[who take] practicum right now . . . already have jobs lined up. . . . So I think it’s definitely really 
beneficial when universities offer that kind of internship or experience because then that’s exactly 
what employers want.” Furthermore, students expressed a keen awareness of how living in an 
increasingly globally interconnected and diverse world affects what job skills are necessary: 


“So as more and more of the country becomes global, where . . . so many people [are] coming here for jobs 
and working in the United States, but also multinational companies . . . are dealing with so many other 
countries as well, having those engaged learning experiences in college . . . learning about other cultures, other 
races, even gender studies . . . is at least somewhat of a preparation for the reality . . . that when you get out 
into the quote ‘real world’ or the working world . . . you’re not going to have this little homogenous picture of 
one type of person.” 


“T found that having a dynamic worldview is really important, but also cultural competency . . . [is] huge 
right now... . [JJust having different perspectives and having the . . . experience of living in a different place 
where you are a minority working with different people of different socioeconomic backgrounds, and . . . being 
a leader, having leadership skills, but . . . also working in a team.” 

“{Wyhen you get an education, you learn more respect, like morals and values, and how to . . . use all that 


in your work field, to understand how people think.” 


Finally, students emphasized the importance of having a strong work ethic, commitment to getting 
an education, and dedication to finishing a degree as behaviors that employers seek in employees: 


° Project researchers have edited the student quotes for clarity and readability. We have used ellipses to indicate omitted language. 
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“A diploma .. . shows that you will work hard and. . . show up every day, and it shows how hard you 
are going to work. . . . [I]t shows that you’re a motivated person.” 


“T guess overall, they’re just looking to see if you have that dedication to go for four years and get a degree 
and see if you can just have the ability to comprehend . . . different things.” 


“Determination. You have to be one that . . . stands out. . . . So, you have to be determined to get to the 
end line . . . to get your degree.” 


{The diploma shows] that you can sit down for at least four years and complete a degree and [know] how 
to focus to graduate. . . . [Y]ou can complete something that you set out to try to do.” 


The learning outcomes these students value correlate strongly with the ELOs. In addition, students’ 
views closely align with employers’ views about what is most valuable in potential employees, 
based on findings from a 2013 survey of employers conducted on behalf of AAC&U.’ The survey 
results indicate that employers believe colleges should place more emphasis on a variety of key 
learning outcomes, including critical thinking, complex problem solving, communication skills, 


and the ability to apply learning in real-world settings. 


According to that employer survey, 95 percent of employers agree (and 57 percent strongly agree) 
that “their company puts a priority on hiring people with the intellectual and interpersonal skills 
that will help them contribute to innovation in the workplace.” Employers “place the greatest 
degree of importance on the following areas”: 


>» Ethics: “Demonstrate ethical judgment and integrity” (96 percent important, including 76 
percent very important) 


> Intercultural Skills: “Comfortable working with colleagues, customers, and/or clients from 
diverse cultural backgrounds” (96 percent important, including 63 percent very important) 


>» Professional Development: “Demonstrate the capacity for professional development and 
continued new learning” (94 percent important, including 61 percent very important)® 


Our focus groups suggested similarities between the viewpoints of students and employers related 
to work ethic and values, cultural competency, and professional knowledge. However, these 
similarities matter little if students don’t understand how particular learning experiences are 
intended to promote the development of the skills and competencies they and their future 
employers agree that they need. To better understand how students perceive the effectiveness of 
particular learning environments, we asked focus group participants to describe the contexts and 
actions that most affect them as learners. As the next section shows, while the students occasionally 
described specific high-impact practices, they more often described the broad characteristics of 
these experiences—suggesting that what matters most to students isn’t whether experiences are 
labeled “high-impact,” but how well these experiences are implemented.’ 


7 Hart Research Associates, It Takes More Than a Major: Employer Priorities for College Learning and Student Success (Washington, DC: 
Association of American Colleges and Universities, 2013). From January 9 to 13, 2013, Hart Research Associates conducted an 
online survey of 318 employers whose organizations have at least twenty-five employees and report that 25 percent or more of 
their new hires hold either an associate’s degree from a two-year college or a bachelor’s degree from a four-year college. 
Respondents were executives at private sector and nonprofit organizations, including owners, CEOs, presidents, C-suite-level 
executives, and vice presidents. For the complete report, see http://www.aacu.org/leap/documents/2013_EmployerSurvey.pdf. 


* Hart Research Associates, It Takes More Than a Major, 4-6. 


° Kuh, High-Impact Practices, 20 (see intro., n. 1). As Kuh notes, “to engage students at high levels, these practices must be done well.” 


(emphasis in original). 
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What learning experiences do underserved students value in developing the skills and 
competencies they view as important to employers? 

As detailed in part I, our quantitative findings support previous research showing that students’ 
engagement in high-impact practices is beneficial to their learning and success.'” But as George 
Kuh states, high-impact practices “take many different forms, depending on learner characteristics 


*11 In examining the “equity effects” of high-impact 


and on institutional priorities and contexts. 
practices on student perceptions of learning, it is important to explore how students, especially 
traditionally underserved students, actually define high-impact learning in its different forms. 


These views can help shape campus designs for effective learning experiences. 


During the focus groups, we asked students a series of questions that addressed their participation in 
engaged learning experiences.'? We asked, for example: “Thinking about your experiences as a 
student, how would you describe what it means to be engaged in your learning?” and “In your 
college experience so far, have there been certain activities or situations (inside or outside the 
classroom) that allowed you to be more engaged in learning?” 


Students most frequently described four types of activities that engaged them to a high degree: 
group work, application of knowledge, interaction with peers, and real-life connections. 


“T never felt more engaged than when I was in group work. . . . [I]f you can’t contribute your knowledge 
and put it to the table the same as everyone else in a group setting with you, then . . . you’re missing out, 
and you just are learning what you want to learn and not exposing yourself to the different perspectives and 
the different alternatives.” 


“{When w]e get into groups and then share ideas with our group members, that’s when I learn . . . when I 
can talk to others about . . . the different material in class rather than just listening to a teacher.” 


“I learned that I work better and retain better in group settings because you bounce . . . ideas off of each other or 
you bounce . . . information off of each other so you’re not only receiving information, but you’re giving it back.” 


“{W]hen you’re in an engaged environment . . . not only is the professor helping you, but you are also 
receiving help and learning how to do something with your fellow peers. [An] engaging environment is when 
the professor and the students . . . collectively as a whole are willing to. . . work for. . . a certain outcome.” 


Students also emphasized the ability to apply knowledge and not just passively receive information: 


“I think it has to do with not just receiving the information, but understanding it, and . . . actually . . . being 
able to apply it. . . . You can receive all the information in the world, but it doesn’t mean you understand it.” 


“You learn it, you apply it, and then you explore it.” 
Students seek real-world application as evidence of engaged learning: 


“T think projects with real-world implications [are the most engaging]. Iwas . . . lucky enough to have more 
than a couple of classes where we did group work, where the outcomes of our projects at the end of the term 


See, for example, Kuh, High-Impact Educational Practices, Brownell and Swaner, Five High-Impact Practices (see intro., n. 5). 


" Kuh, High-Impact Practices, 9. 


* During the focus groups, the researchers referred to the learning experiences as “engaged learning” instead of “high-impact practices” 
in order to minimize the use of jargon that may be unfamiliar to students. 


'5 See the focus group script in appendix D for additional examples of questions related to engaged learning experiences. 
group scrip PP p q gag g exp 
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made a real-world difference where we were working at the nonprofit, at the library. . . . [All of the systems 
that we had researched and designed actually went someplace, and that was incredible.” 


“{When you do] something hands-on . . . that’s when you walk away with . . . a meaningful . . . benefit to 
what you’re learning in the classroom.” 


Students believe that relevancy to their lived experiences promotes engagement with learning: 


“{The professor] would apply [classroom material] to . . . our daily lives... . [H]e would use... TV shows 
or use radio or something else that . . . we actually listened to instead of just . . . [saying] ‘here’s Aristotle’ 
and. . . all these old people that are already dead that . . . none of us knew or know anything about.” 


“T have a professor . . . [who] does not require a textbook at all, and I have not missed one of his classes . . . 
I wake up early to go, and it’s because what he teaches us is relevant to our lives. I think that’s the most 
important thing, because a lot of students here, they go to class, and what they’re learning has nothing to do 
with what they want to do in life, so I feel like relevancy . . . kind of engages you more on the subject.” 


“There’s really no way to engage with someone who is talking at you instead of talking with you because 
there’s no room for the students’ experience to have value or there’s no room for you to question the professor. . . . 
[That approach] completely devalues the student’s time. It devalues their personal lived experiences.” 


Students noted the ability to connect out-of-classroom learning experiences and classroom 


activities as a component of engaged learning: 


“T think [engagement in cocurricular experiences] has enhanced my learning personally because without me 
having to go out and take these opportunities and go to seminars and lectures and learn something out of it, 
I’m taking away the knowledge that I gained and being able to apply a new skill that they might not teach 
in a classroom. And the seminars . . . [are] critical thinking at its finest. . . . [Y]ou get to hear a lot of 
opinions going back and forth or different discussions about different things that are going on in the world or 
even locally, and it’s just fascinating at times.” 


“T learned the absolute most from my research project . . . with the professor as well as [from] my capstone 
experience because both of those fostered independent learning. . . . And L feel like that’s the most important thing . . . 
not really befing] told the answer, but rather be[ing] told how to figure it out . . . because I feel like it [the research 
project] was a lot of building blocks... . [Y]ou’re just kind of constantly building on your learning.” 


‘Pve learned to ask questions . . . [in] group settings; and talking to professors and being involved in the 
community, you learn how to ask better questions, more focused questions that improve your communication skills.” 


“With the internship, I really appreciate the experience of being able to bring [my experience] back to the 
classroom to share with others.” 


Students also identified access to and participation in support networks as important to their 


engagement with learning: 


“{As p]art of my first-year experience, we formed . . . a mentor group... .. A bunch of us... [are] all taking 
some of the same classes, and so we found people that were also taking those classes and formed . . . study 
groups... and that really helped. . . . [A] lot of the same people are still in. . . them.” 


Or, as another student simply stated, “What got me engaged was having a mentor.” 


The focus group findings indicate that high-impact or engaged learning can take place in multiple 
settings, whether designed specifically to support high-impact practices or representative of typical 


28 | ASSOCIATION OF AMERICAN COLLEGES AND UNIVERSITIES 


classrooms. Moreover, focus group participants did not indicate the need for elaborate designs. Rather, 
they pointed to low-cost dimensions of high-impact practices—such as reciprocal engagement with 
peers and faculty in the learning process—as what makes these practices effective. Students described 
engaged learning experiences by the qualities that make them effective, rather than by a particular 
name or label. As the quotations above exemplify, students may not have referred to their “learning 
community” experiences or their “collaborative group work” experiences. But they did cite the value 
of being in educational environments that encouraged them to interact with others, explore differing 
opinions, apply knowledge in real-world settings, incorporate their lived experiences into their 
learning, and participate in support networks.'* 


What factors do underserved students identify as barriers or obstacles to their 

participation in high-impact learning experiences? 

Despite the benefits of participating in high-impact practices documented in this report and in 
previous research,” participation in these practices remains relatively low, as discussed in part I of 
this report. Given the well-documented advantages of these types of learning activities, higher 
education stands to benefit from examining access to high-impact practices more closely. 


To gather information about this topic, we asked focus group participants, “What would you say 
most influences your decision to seek out and participate in specific engaged learning experiences, 
for example service-learning, undergraduate research (research with a faculty member), study 
abroad, or internships?” Students’ responses to this question generally focused on barriers related to 
a lack of advising or guidance about what high-impact practices are or why they are important, and 
an inability to commit time to these experiences due to the constraints of busy lives. 


First, students desire relationships with adults who care about their engagement with learning and 
advisors who can provide information that will help them stay on course: 


“[T]he best professors that I’ve had . . . not only are they knowledgeable, but . . . they’re interested in 
your learning.” 


“{E]ven though they [my parents] are smart people, they weren’t college people, and so they didn’t seem to 
have the forwithal [sic] to tell me this [exposure to learning opportunities] is part of why college is different. . . . 
[I]f I had some kind of mentor to . . . navigate that, it would have been a little bit easier, and I probably 
would have been done earlier, but I didn’t know.” 


“I mean she [the professor] gave me positive feedback. She actually gave me the materials that I was not 
presented with in high school. So I was actually able to go out, do my studies and do my research, and do 
what need[ed] to be done to get my essays done.” 


“T’ve tried doing a study abroad, but . . . it’s convoluted and doesn’t make much sense . . . so it gets put on 
the back burner just because it’s . . . one more thing to try to figure out on your own.” 


'* Others have written about many of these factors as essential aspects of what makes high-impact practices effective. See, for example, 
Kuh, High-Impact Educational Practices; Alma R.. Clayton-Pedersen and Ashley Finley, ““What’s Next? Identifying When High-Impact 
Practices are Done Well,” afterword to Five High-Impact Practices: Research on Learning Outcomes, Completion, and Quality, by Jayne E. 
Brownell and Lynn E. Swaner (Washington, DC: Association of American Colleges and Universities, 2010), 53-7. 


' See, for example, Kuh, High-Impact Educational Practices. 
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“Tt was either ‘you keep up or you’re gone. I don’t care.’ P’m not looking . . . necessarily [for someone] to 
hold my hand, but if I’m coming to you for help, I’m expecting to get help from you, and they [faculty 
members] would pass me off onto another student that was struggling just as much as I was.” 


For many underserved students, competing priorities (e.g., work) hinder their participation in 


engaged learning activities: 


“T would love to do . . . internships and stuff that . . . all my professors keep talking about . . . but the fact is 
...Ican devote . . . forty hours of unpaid time [through an internship] on top of school, and then . . . 

what’s going to pay for school? Working. So I have to find another job on top of that. So. . . either I can 
clone myself, which hasn’t been developed yet, or I just drop the internship and then figure it out later. . . . 


a3 


“So, I would work at nights and come to school during the day, and. . . if I wasn’t doing one, I was doing 
the other. And the time between, I was studying. Now .. . I’m so busy with classes . . . and trying to just 
pass everything so I can graduate . . . I don’t want to risk it by going and doing too many extracurriculars 
and then maybe not passing a class and having that put me back.” 


“Having so many other classes, I don’t have time to do my homework and work and then go volunteer too.” 


Students also suggested the need for greater transparency about how high-impact activities connect 
to their learning, specifically when it comes to service-learning experiences. They expressed the 
desire for additional guidance about how to identify high-quality high-impact experiences: 


“We just kind of stood there [at the service-learning location] and we’re like, okay. We’re not really learning 
anything. It should have been something to do with [a] class I was taking at least or something.” 


“T feel like it [service learning] didn’t really have anything to do with my academics, but it was something to 


” 


pass the time... . 


“T think a couple of barriers that exist are, one, course catalogues or when students are trying to register for 
ple o g ying fo register | 


courses. If you’re lucky, you get maybe a two-sentence blurb about what the course is about. So. . . whether 
or not you get that engaged learning experience . . . [is] luck of the draw based on who the professor is... .” 
Discussion 


In part I, we examined the different rates of participation in high-impact practices among underserved 
students and students from traditionally advantaged groups, and the quantitative relationship between 
participation in particular high-impact practices and students’ perceptions of their own learning (across 
all students and among students from various underserved and traditionally advantaged groups). ‘This 
section complements the previous section and brings deeper meaning to the quantitative findings by 
incorporating student voices into the discussion of high-impact practices. 


Responses from focus group participants suggest that there is strong alignment between students’ 
views of the skills and competencies that matter for the workforce, employers’ views as expressed in 
AAC&U’s 2013 employer survey, and AAC&U’s Essential Learning Outcomes.'° This alignment 
provides an excellent opportunity to promote liberal education and high-impact practices as 
potential means of effecting these outcomes. In addition, some responses indicate that students are 
not oblivious to the types of experiences that engage them with their learning environments. They 
value group work (when designed well), interaction with their peers, real-life connections to 


'¢ Hart Research Associates, It Takes More Than a Major. 
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classroom content, and the opportunity to apply their knowledge. They do not require elaborate or 
expensive high-impact practices, and instead highlight the relatively low-cost dimensions of high- 
impact practices as significant means of promoting student engagement. 


Some focus group participants also noted significant barriers and obstacles to participating in 
high-impact practices, including limited time and money, competing priorities, and inadequate 
social support networks. These barriers are realities for more and more students, given the growing 
number of students who are members of one or more groups that have been traditionally 
underserved in higher education. As we found in part I, engagement in multiple high-impact 
practices has a positive effect on students’ perceptions of their own learning. Therefore, campus 
practitioners should make a considerable effort to diminish the barriers to participation in these 
practices and to design structures that promote greater access to them. 


If these focus group findings hold true on campuses, what do they mean for approaches to 
integrating high-impact practices into the curriculum and cocurriculum? How can educators 
design an inclusive educational environment where all students have the opportunity to participate 
in high-impact practices? And what can educators learn from their students to better inform their 
work to improve student success? 


Student voices are essential when examining what works to improve student success in higher 
education. While our focus groups included too few participants for the findings to be generalizable, 
the sessions unearthed comments that were consistent across multiple campuses. These comments 
provide insight into the student experience, and we hope they will spur interest in replicating 
similar discussions that can likewise provide valuable data to inform decisions about student 
learning. Whenever possible, campus leaders should incorporate student voices into their efforts to 
support student success. 
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CONCLUSION 


Guiding Questions for Advancing Campus 
Efforts to Improve Learning and Promote 


Success for Underserved Students 
=x* * w&e we kK *Ke hh jKrmhUcCY 7K Ke Ke Ke Kh hUK 


Campus leaders are beginning to ask a critical but complex question: “How can we construct or 
restructure our educational environments to better serve the diverse needs of today’s students?” In a 
context where the increasing national focus on college completion rates threatens to overshadow 
the importance of ensuring that all students have access to high-quality learning experiences, there 
are no easy answers and no truncated timelines. Addressing this question will require a long-term 
commitment to a process of continual inquiry and self-examination that can provide an informed 


understanding of campus cultures, educational contexts, and student experiences. 


Both existing institutional data and student voices should be key aspects of such an investigative 
process. Below are some suggestions for campus leaders as they initiate discussions among faculty, 
administrators, and students about how to improve the learning environment for all. While not 
definitive, this list can serve as a starting point for campus conversations about the findings 


summarized in this report. 


>» The research presented in this publication has significant implications for how colleges and 
universities plan and structure high-impact practices across the curriculum and cocurriculum. 
On campuses where these experiences are centralized within the first or senior year (or 
perhaps both), educators may be missing opportunities to optimize learning gains by fully 
distributing engaged learning practices across the entire undergraduate experience. More 
intentional practices within the sophomore and junior years can connect high-impact learning 
in the first and senior years, resulting in expanded and integrative cornerstone-to-capstone 
experiences. How can you use the data presented here and similar data gathered on your own 
campus to structure better learning environments? 


> This research highlights the need for greater intentionality in articulating the value of high- 
impact practices and the ways they contribute to helping students gain the skills and 
competencies that prepare them for life and work. Campus practitioners should explore the 
degree to which their students are aware of the connection between these skills and 
competencies and institutional learning outcomes. By intentionally engaging with their 
students about this connection, educators could promote higher levels of student engagement. 
How and to what degree does your institution help increase students’ awareness of institutional 
learning outcomes as a way to foster engagement and to promote relevancy? 
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ates 


>» This research points toward the need to make high-impact practices pervasive on campus. To 
accomplish this, college and university leaders will need to engage all faculty in developing 
and implementing high-impact practices—including non-tenure-track faculty (many of whom 
are employed on a part-time basis). Because non-tenure-track faculty now represent more 
than two-thirds of all faculty at colleges and universities nationwide,? this group contributes 
significantly to the undergraduate learning experience, as do the many student affairs 
professionals who guide learning experiences during the significant amount of time students 
spend outside of the classroom. How can the findings presented here and related findings on 
your campus influence professional development for all faculty and staff? 


>» This research underscores the value of including multiple and varied sources of data in the 
inquiry process. Evidence related to students’ experiences (such as that collected through 
the surveys and focus groups described here) is a valuable source of information. But ideally, 
this evidence should be combined with data collected through direct assessments of the 
products of students’ actual work, which can be captured through assignments embedded 
in the curriculum and cocurriculum. How are you triangulating useful evidence to inform 
decision making about approaches to learning in order to improve student success? 


Designing and sustaining an educational environment that results in high-quality learning experiences 
and equity in student achievement requires a genuine commitment of time and effort, an exploration 
of the student experience, institutional assessment and data analysis, and the ability and courage to 
implement change. We hope that the questions outlined above and the research and methodology 
described in this report encourage campus practitioners to further explore and improve their own 
strategies for advancing student success and expanding engagement in high-impact practices. 


' American Association of University Professors, AAUP Contingent Faculty Index 2006 (Washington, DC: American Association of 
University Professors, 2006), 6. http://www.aaup.org/sites/default/files/files/AAUPContingentFacultyIndex2006.pdf. 
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Appendix A 
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To encourage campus practitioners to engage in more in-depth analysis of high-impact practices, 
Association of American Colleges and Universities researchers invited the Center for Urban Education 
(CUE) at the University of Southern California to develop a toolkit for assessing individual high-impact 
practices based on CUE’s Equity Scorecard™., While the authors of Assessing Underserved Students’ 
Engagement in High-Impact Practices did not use the Equity Scorecard™ in their research, the 
toolkit provides a framework campus leaders can use to examine individual high-impact practices from an 
“equity-minded” perspective based on who participates in these practices, who has access to them, and 
what impact participation has on identified outcomes. Focused on advancing racial and ethnic equity, 

this toolkit helps practitioners examine institutional approaches so that all students can benefit from 
high-impact practices. 


Assessing Equity in High-Impact Practices Toolkit’ 
Developed by the Center for Urban Education at the University of Southern California 


Estela Mara Bensimon, professor of higher education and codirector of the Center for Urban 
Education at the Rossier School of Education, University of Southern California 


Alicia C. Dowd, associate professor of higher education and codirector of the Center for Urban 
Education at the Rossier School of Education, University of Southern California 


Debbie Hanson, project specialist at the Center for Urban Education and Equity Scorecard 
facilitator, University of Southern California’ 


Since its founding in 1999, the Center for Urban Education (CUE) at the University of Southern 
California has helped thousands of college and university professionals—from presidents to 
academic counselors to faculty—take steps in their daily work to reverse the impact of historical 
and structural disadvantages that prevent many students of color from making their way in higher 
education. Through an “action research” approach, CUE empowers faculty and administrators to 
interrogate the impact of their own practices and policies so they can better understand and then 
reduce race-based inequities on their respective campuses. This approach leverages the strengths of 
academic culture, including a commitment to inquiry, collaborative decision making, and 
professional ethics centered on supporting student success. Participatory action research carried out 


The Center for Urban Education at the University of Southern California retains the copyright to the Assessing Equity in 
High-Impact Practices Toolkit and authorizes AAC&U to produce copies and reproductions of this material. 


iy 


The authors wish to acknowledge Dominic Alpuche for creating CUE’s data analysis tools, Deanna Cherry for designing 
facilitation approaches, Svetlana Levonisova for developing the syllabus analysis tool, and Greg Steirer and Emily Ogle for their 
editorial assistance. 
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by teams of professionals enables institutions to take greater ownership of sustained efforts to 
change cultural norms and fundamental practices for the benefit of students of color. 


By providing intuitive, interactive tools that help illustrate and “make real” the racial inequities that 
exist on campuses, CUE creates opportunities for practitioners to become more “equity-minded” in 
the ways they interpret and support students’ success. Equity-mindedness is a concept that locates 
responsibility for the educational success of students of color in institutional practices rather than in 
student deficits. CUE’s toolkits are thus designed to change the cognitive framework used by 
individual practitioners to approach learning outcomes, leading practitioners to examine not what 
students are doing right or wrong, but what they themselves are doing and how their actions both 
reflect and give rise to the broader institutional culture. CUE believes that when practitioners 
recognize that their practices are not working and then participate in deliberate learning opportunities to develop 
the new knowledge necessary to support equity-minded practices, they can make a substantial difference in 
educational outcomes for racial and ethnic groups that experience inequities in college access and 
success. (For more information on CUE?’s theory of institutional change, tools, and publications, 
please visit http://cue.usc.edu/.) 


The Assessing Equity in High-Impact Practices Toolkit suggests a process practitioners can follow to 
become more equity-minded in their use of high-impact practices (HIPs). The toolkit is designed 
to help practitioners ensure that students from all racial and ethnic groups are taking equal 
advantage of these important practices, which are known to greatly improve educational outcomes 
for students overall. The toolkit provides methods to assess the provision of high-impact practices 
according to three criteria for racial and ethnic equity: equal representation, equal access, and equal 
impact. It helps practitioners answer such questions as the following: 


>» What high-impact practices do we use? 


>» Is access to and participation in high-impact practices equitable across different racial 
and ethnic groups? 


> In what ways are high-impact practices responsive to the cultures and experiences of 
students from different racial and ethnic groups? 


>» Dowe continuously monitor racial and ethnic equity in high-impact practices, 
and in what ways? 


To assist practitioners in answering these and other questions—and ultimately in developing 
more culturally sustaining practices and interventions—the toolkit outlines a six-step process, 
beginning with the selection of a particular HIP for assessment and ending with the creation of 
equitable benchmarks. 


Step 1: Select a High-Impact Practice for Assessment 

Instructors, administrators, counselors, and campus leaders should be involved in using the toolkit to 
conduct inquiry. Begin by inventorying all the HIPs taking place on your campus (see figure A.1). 
After doing so, choose one HIP to assess with respect to equity. In choosing a particular practice for 
assessment, you might consider the following questions: In which of these practices is the college/ 
university making the greatest investment? Which is intended to have the broadest impact in 
relation to long-term outcomes like increased graduation rates? 
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Figure A.1 
Choosing a HIP 


Which of the following high-impact practices are taking place on your campus? Circle all that apply. 
Then choose one of the circled items for detailed assessment. 

First-Year Experience/Seminar Learning Communities 

Common Intellectual Experiences Writing-Intensive Courses 


Collaborative Assignments and Projects Internships 


Diversity/Global Learning Undergraduate Research 


Service Learning, Community-Based Learning Other: 


Step 2: Gather and Analyze Data 

After choosing a HIP for further assessment, gather data detailing student representation in the 
program (disaggregated by race and ethnicity), criteria for access to the program, and program impact 
on those who participate (also disaggregated by race and ethnicity). To enable equity-based 
comparisons, assemble additional enrollment data for the institution as a whole. 


Figures A.2 through A.4 provide examples of what such data might look like and how institutions 
might go about analyzing them. The data these figures offer pertain to an undergraduate research 
program at the fictional Downtown College in Washington, DC. Downtown College explicitly designed 
the undergraduate research program in question to increase completion rates among participants. 


Figure A.2 pairs data on overall student enrollment disaggregated by race/ethnicity with equivalent 
data on participation in the undergraduate research program. By examining these numbers together, 
practitioners can see that African Americans, Latino/as, and Native Americans are underrepresented 
within the undergraduate research program compared to their overall representation in full-time 
enrollment at Downtown College. 
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Figure A.2 
Undergraduate Research at Downtown College—Analyzing Representation 


Equal Representation 


Which groups are experiencing equal or unequal representation? 
Compare the representation (%) in tables A and B and write 


= if there is no difference 
if there is a smaller % in the HIP 
T if there is a /arger% in the HIP 


Figure A.3 examines one criterion for participation in the undergraduate research program. Because 
only full-time students are eligible for the program, a comparison of data on both full-time and 
part-time student enrollment, disaggregated by race/ethnicity, reveals whether this policy adversely 
affects students of color. The comparison indicates that African Americans, Latinos/as, and Asians 
make up a higher share of part-time than full-time enrollment. The policy thus disproportionately 
benefits the white segments of Downtown College’s student population. 


Figure A.3 
Undergraduate Research at Downtown College—Analyzing Access 


Selection criteria: Students must be enrolled full time. 


Part-Time 
Students 


Equal Access 


Full-Time 
Students 


How might this selection criterion affect students differently? 


Which racial/ethnic student groups are under- or overrepresented in part-time enrollment? 
Compare the representation (%) in tables C and A and write 


= if there is no difference 
4 if there is a smaller % enrolled part time 


T if there is a /arger% enrolled part time 
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The criteria you choose to analyze could represent formal or informal ways of selecting students for 
participation in a HIP. For example, students might need a certain grade point average (GPA) to 
participate, in which case you would compare by race/ethnicity the percentages of students who 
have that GPA to the percentages of enrolled students. Or you might know anecdotally that only 
students in certain majors find out about the opportunity, representing an informal selection 
criterion. You could then compare by race/ethnicity the percentages of students majoring in those 
disciplines to the percentages of enrolled students to determine whether the process for making 
students aware of the opportunity is disadvantaging certain groups. 


Figure A.4 shows six-year graduation rates (disaggregated by race/ethnicity) for all students who first 
enrolled in 2005, as well as for students first enrolled in that year who participated in the 
undergraduate research program. By subtracting the graduation rate for all students (row D) from the 
graduation rate for participants in the HIP (row E), practitioners can see at a glance the extent to 
which graduation rates differ for each group. In this case, the data reveal that African American 
students’ participation in the program correlates to no significant increase in their graduation rate, 
compared to that of the African American population overall. For other races/ethnicities, participation 
in the program correlates to much higher graduation rates—but with whites and Asians seemingly 
benefitting more than Latino/as. Causal relationships between HIP participation and outcomes may 
be difficult to establish without statistical tests for significance and analytic techniques that take into 
account prior differences between participants and nonparticipants. However, this simple approach to 
examining equal impact allows faculty and administrators to see whether patterns of inequity persist 
among students who have participated in high-impact practices. 


Figure A.4, 
Undergraduate Research at Downtown College—Analyzing Impact 


Equal Impact 


Which racial/ethnic groups are experiencing greater or lesser impact? 
Subtract the numbers in table D from those in table E, and enter those numbers in the boxes. In front of 
each number, write 


=  ifthere is no difference 


4 if the difference in graduation rate is a negative number 


T if the difference is a positive number 


The figures above offer just one example of data analysis directed toward assessing a HIP’s role in 
advancing equity. Although this kind of analysis will almost always require certain data sets (such as 
data on enrollment and participation), the other data sets required will vary according to the HIP 
being assessed. For any given HIP, the data used to measure impact should align with the HIP’s 
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intended goals. Assessing the impact of a writing-intensive course, for example, might mean 
comparing grades earned in later writing-intensive courses by students who did and did not 
participate in the HIP. Similarly, assessing the effects of internships might mean comparing post- 
degree job placement rates for students who did and did not participate. Even if improved 
graduation rates are not a professed goal of a specific HIP, examining these rates might reveal 
whether participation in HIPs correlates with differences in success among students of different 
races/ethnicities. A complete template for analyzing data with respect to equity in HIPs is available 
at www.aacu.org/assessinghips/. 


Step 3: Interrogate Policies and Practices 

Data disaggregated by race/ethnicity can help those using the toolkit identify racial and ethnic 
inequity with respect to both participation in HIPS and their outcomes. These kinds of data typically 
elicit two types of responses from observers: (1) deficit-minded explanations—that is, explanations that 
locate the reason for the equity gap in student characteristics (i.e., blaming the students); or (2) generic 
solutions based on preexisting assumptions about the nature of the problem. 


CUE recommends a different approach. Rather than assuming that the reasons for the equity gap are 
already well known, treat the gap as an indeterminate situation or puzzle that calls for further 
investigation. Likewise, instead of focusing on student characteristics as the cause of the inequity, 
frame the gap as an indication of institutional failure. To help structure the investigation in these 
ways, begin by asking the following questions: 


>» In what ways are my/our teaching practices failing to achieve successful outcomes for 
African American, Latino/a, or Native American students, or for other student groups that 
the data might reveal are not benefitting equally? 


>» In what ways do our policies and practices have an adverse impact on these students? 


> In what ways do our data fail to alert us to equity gaps for these students? 
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Figure A.5 provides examples of both equity-minded and deficit-minded explanations for why 
African American, Latino/a, and Native American students are underrepresented in the honors 


program of the fictional Downtown College. 


Figure A.5 
Deficit-Minded Explanations of Equity Gaps and Equity-Minded Questions 


How do we do Do we regularly evaluate 
undergraduate student representation in 
research here? undergraduate research? 


Research shows | (BJaulelielMarel-te 


that they are less | MIRCEMELsolats 
motivated. 


How do we encourage or reward faculty who include 
underrepresented students in research? 
They didn’t have high enough GPAs 
or ACT/SAT scores and did not How many students can the Is undergraduate 
qualify for courses that include undergraduate research research culturally 
undergraduate research. program accommodate? inclusive? 


>| “a 


It doesn’t fit with their schedules How do we let students know | | What are the eligibility 
because they work too many hours. about undergraduate research? requirements? 


Does our campus offer undergraduate 
Se/Vijaye\grelse) | | research opportunities funded by the 
Questions National Science Foundation for 
minority students? 


They lack the habits needed to do 
undergraduate research. 
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Figure A.6 offers a template designed to help practitioners develop institution-specific questions 
about the equity gaps revealed in step 2. 


Figure A.6 
Template for Developing Questions about Equity Gaps 


Fill in the circle below with the resources (departments, documents, people, and policies) that are 
directly involved with equity gaps. Next, fill in the area outside the circle with questions about the 
resources you identified that will help you shed light on how your institution’s policies and practices 
may be contributing to equity gaps. 


Departments, people, events, documents, 
and assessment practices that are directly involved 
with equity gaps 


Equity Gap 
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Step 4: Plan Inquiry Activities 

After you have developed specific questions about the policies and practices that contribute to equity 
gaps within the HIP you selected, you will need to develop a set of strategies to answer these 
questions. Plan inquiry activities (such as interviews, observations, and document reviews) to help you 
better understand and evaluate the day-to-day practices and policies associated with the equity gaps in 
question. By evaluating your institution’s high-impact practices through an equity-minded lens, you 
will be able to determine which activities are effective in supporting the needs of students of color, as 
well as which activities could be changed to improve equity at your institution. Figure A.7 provides a 
list of possible inquiry activities. Figures A.8 and A.9 provide examples of inquiry activities for two 
different practices, with figure A.8 listing inquiries and prompts focused on undergraduate research 
programs and figure A.9 providing a sample protocol for performing a document review on a syllabus. 


Figure A.7 

Types of Inquiry Activities 

The Center for Urban Education has developed a range of assessment protocols to assist 
practitioners in conducting inquiry into equity gaps. Visit http://cue.usc.edu for more information. 


; » Syllabi reviews 
Document Analysis : 
Test/assessment reviews 


Observation Guides Classroom observations 


Interview Protocols Faculty interviews on classroom practices 


Faculty awareness of student support services 
“Getting to Know You,” “I Learn Best...,” and “My Progress” 


VV\iV|\ Vv a 


Surveys 


; Faculty hiring pool, representation, and retention by race/ethnicity 
Data Analysis : 
> Campus climate reports 


>» Analysis of faculty development resources 
Document Analysis (website and handouts) 


>» Analysis of questions posed in course evaluations 


>» Observation of faculty and teaching assistant training 


Observation Guides 
> Observation of faculty development sessions 


>» Administrator interviews on recruitment, hiring, and 
retention strategies and priorities 


Interview Protocols >» Faculty interviews on experiences with recruitment, hiring, 
and retention 


>» Faculty and TA interviews on training experience 


>» Curricular mapping 


Mapping Activity >» Mapping course objectives and lesson plans against 
curricular goals 


>» Visiting campuses with existing, effective developmental 


Campus Visits : 
course practices 


Literature Review » Researching existing best practices in developmental education 
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Figure A.8 
Undergraduate Research—Examples of Inquiry Activities and Related Prompts 


iry Activity | Examples of Equity-Minded Inquiry Prompts 


» Describe how you involve students in undergraduate research 
opportunities. 


>» Describe how you think students learn about undergraduate research 
opportunities. 


>» Are you aware that specific racial/ethnic groups are underrepresented in 
undergraduate research? Why do you think such underrepresentation exists? 


>» In what ways could more students of color be involved in undergraduate 
research? Which of these would you be willing to try? 


In what ways is the content of undergraduate research culturally inclusive? 


Describe how you learned about your undergraduate research opportunity. 


What interactions do you have with your faculty mentor? 
What could be done to involve more students of color? 


> 
> 
>» What did you need to do to apply for the opportunity? 
> 
> 
> 


In what ways do you encourage faculty to involve students of color in 
undergraduate research experiences? 


In what ways can you promote greater attention to equity in HIPs? 


Could the institution collect and report HIP data by race and ethnicity? 


Could the administration set specific participation goals for undergraduate 
research by race and ethnicity? 


> 
» What data do you use to assess equity in HIPs? 
> 
> 


>» Do online resources on undergraduate research opportunities use plain 
language, or do they include terminology that might be confusing for 
students without prior college exposure? What would students need to 
know? 

>» What tone of voice does the website use? 

>» What information does the website provide for students who have 
additional questions? 


>» What are the eligibility criteria? 
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Figure A.g 
Example of Syllabus Review Focusing on Indicators of Culturally Inclusive Practices 


Suggests an environment Yes/No/Somewhat Yes/No 
in which there is genuine If Yes or Somewhat, how so? If Yes, how? Note possible steps: 
respect for students anda 
belief in their capabilities 
If No, should they? Yes/No If No, why not? 


How could they be revised? 


Communicates an expectation Yes/No/Somewhat Yes/No 
Chem poe tine scude ns If Yes or Somewhat, how so? If Yes, how? Note possible steps: 
If No, should they? Yes/No If No, why not? 


How could they be revised? 


Reflects the use of a variety of Yes/No/Somewhat Yes/No 
pedagogical approaches that If Yes or Somewhat, how so? If Yes, how? Note possible steps: 
acknowledge and teach about 
race and racism 
If No, should they? Yes/No If No, why not? 


How could they be revised? 


Demonstrates expectations of Yes/No/Somewhat Yes/No 
reflection Be ames of ; If Yes or Somewhat, how so? If Yes, how? Note possible steps: 
incorporating issues of equity 
into teaching, thinking, and 
practice 
If No, should they? Yes/No If No, why not? 


How could they be revised? 
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Step 5: Identify Culturally Inclusive Practices and Interventions 

After completing the inquiry activities, convene the inquiry team to discuss insights and share possible 
actions that practitioners could take to improve equity. During the conversation, organize these actions 
under the categories of “within reach,” “‘a stretch,” and “beyond reach.” Where applicable, map these 
actions onto key milestones in students’ educational progress at the institution as a way of assessing 
potential impact. For example, by mapping actions onto second-year retention (such as conducting more 
outreach during the first year) or a second year of persistence for transfer students (such as including 
information about the HIP during transfer orientation), you can help pinpoint opportunities for 
intervention as well as help practitioners evaluate the effectiveness of those interventions. 


To ensure a productive group discussion, it is important to emphasize collective assessment rather than 
individual evaluation. The focus should be on framing the problem, experimenting, and generating 
solutions, not on assessing the performance of any particular office or colleague. Keep in mind that the 
findings will represent educational practices that have been shaped over long periods of time by your 
campus’s specific culture and norms. For this reason, participants should try to depersonalize their 
language, even when referring to existing departments or specific documents created by colleagues. 
Always support your interpretations with data, and always refer to the specific language or findings 
collected during the inquiry process. 


Step 6: Implement Actions and Set Equity Goals 

In the final step, select for implementation some of the actions identified in the previous steps. When 
planning the details of implementation, set equity goals as benchmarks for each action. These benchmarks 
will help you both design the implementation plan and assess its success. To set these benchmarks, 
envision what your HIP would look like if it were fully equitable in representation, access, and impact. 
Figure A.10, which continues the examination of Downtown College’s undergraduate research program 
presented in figures A.2 through A.4, provides an example of how to establish these benchmarks. 


Figure A.10 
Undergraduate Research at Downtown College—Establishing Benchmarks 


57 


127 


16 


121 


2,294 


2,615 


2.2% 


4.9% 


0.6% 


4.6% 


87.7% 


100% 


In order to achieve equitable 


participation for African American students in undergraduate research, 
three additional African American students (126 X 2.2%) would need to participate. 


0% 


2.4% 


0% 


7.9% 


89.7% 


(e) 


3 


(e) 


10 


113 


In 2011, 126 students participated in undergraduate research. Assuming that the number of 


participants remains roughly the same in subsequent years, African Americans will need to make up 
2.2 percent of that number (or three students, rounding up from 2.7) in order to achieve 
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representation that is proportional to their representation in the student population. Since no 
African American students participated in undergraduate research in 2011, three more will need to 
participate to achieve this benchmark. Similarly, the number of participating Latino students will 
need to double, from three to six. 


Using raw numbers in discussions about addressing equity gaps is often empowering because 
goals sound manageable and attainable when they translate into small numbers of students. 
Faculty and staff who understand gaps in terms of raw numbers are more likely to feel that the 
equity goal is achievable. 


Once you have formulated a plan with associated benchmarks, create an equity goal statement 
summarizing key details. These details should include a goal summary, a timeline, an implementation 
plan, and a description of the desired impact. Figure A.11 offers an example of such a goal statement 
for Downtown College. A more detailed template is available online at www.aacu.org/ 
assessinghips/. 


Figure A.11 
Undergraduate Research at Downtown College—Equity Goal Statement 


Goal: Increase the number of African Americans involved in undergraduate research 


Goal Summary: Downtown College will increase the participation of African American students in its 
undergraduate research program. Beginning with the 2011-12 school year, we plan to increase the 
number of African American participants to at least three students per year. An increase of three 
students would represent parity with African American students’ 2.2 percent share of total 
undergraduate enrollment. 


Timeline: We will close our equity gap in African American participation in undergraduate research by 
fall 2013. 


Implementation Plan: To achieve this equity goal, we will conduct interviews with students and faculty 
and review marketing materials (including the program’s website) and eligibility criteria for 
undergraduate research programs. The purpose of the review will be to identify ways in which our 
current practices, policies, and structures might present obstacles to African American students’ 
participation in undergraduate research. 


Planned Impact: African American students will be represented in the undergraduate research program 
in the same proportion as they are represented in the overall student body. 


Conclusion 

The action research model developed by the Center for Urban Education presupposes that 
professionals in an academic community are committed to the success of their students.’ The inquiry 
methods described in this toolkit can tap into the potential of higher education professionals to 
become agents of equity—within HIPs, through their own areas of responsibility, and in the 
institution at large. 


3 For more on the Center for Urban Education’s action research model, see Estela Mara Bensimon and Lindsey Malcom, eds., 
Confronting Equity Issues on Campus: Implementing the Equity Scorecard in Theory and Practice (Sterling, VA: Stylus, 2012). 
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Appendix B 
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National Survey of Student Engagement Deep Learning and Self-Reported 
Gains Scales 
National Survey of Student Engagement (NSSE) scales are composed of the items identified below.' 


Deep Learning Scales 

According to NSSE, “Deep approaches to learning get at the underlying meaning of an issue, not just 
surface knowledge, emphasizing a commitment to understanding and reflecting on relationships 
between pieces of information rather than rote memorization. Such learning involves applying 
knowledge to real-life situations and successfully integrating previous learning.”? To measure deep 
approaches to learning, NSSE asks students about their engagement in the following activities: 


> analyzing the basic elements of an idea, experience, or theory, such as examining a particular case 
or situation in depth and considering its components 


» synthesizing and organizing ideas, information, or experiences into new, more complex 
interpretations and relationships 


>» making judgments about the value of information, arguments, or methods, such as examining 
how others gathered and interpreted data and assessing the soundness of their conclusions 


>» applying theories or concepts to practical problems or in new situations 
work[ing] on a paper or project that required integrating ideas or information from various sources 


includ[ing] diverse perspectives (different races, religions, genders, political beliefs, etc.) in class 
discussions or writing assignments 


Vv 


put[ting] together ideas or concepts from different courses when completing assignments or 
during class discussions 


>» discuss[ing] ideas from ... readings or classes with faculty members outside of class 


>» discuss[ing] ideas from ... readings or classes with others outside of class 
(students, family members, coworkers, etc.) 


> examin[ing] the strengths and weaknesses of [one’s] own views on a topic or issue 


> tr[ying] to better understand someone else’s views by imagining how an issue looks from 
his or her perspective 


> learn[ing] something that changed the way [one] understand|[s] an issue or concept 


” 
> 


National Survey of Student Engagement, “Measurement Scales, Component Items, and Intercorrelation Tables (NSSE 2011) 
accessed May 17, 2013, http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_InternalConsistency_2011_ 
Intercorrelations.pdf. Bulleted items are quoted directly from survey documents except where indicated by brackets. 


? National Survey of Student Engagement, “Creating Scales and Scalelets,” accessed May 30, 2013, http://nsse.1ub.edu/_/?cid=368. 
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Gains Scales 


NSSE has developed three scales that “explore the degree to which students report having made gains in a variety 


of personal, practical, and general education competency areas as a result of their undergraduate education.’ 


Students indicate how much they believe their education has contributed to their learning gains in the three areas 
described below, and NSSE researchers use those responses to construct the respective gains scales. 


Gains in General Education 


>» writing clearly and effectively 
> speaking clearly and effectively 


>» acquiring a broad general education 


Vv 


thinking critically and analytically 


Gains in Practical Competence 


>» acquiring job or work-related knowledge and skills 
working effectively with others 
using computing and information technology 


analyzing quantitative problems 


Vv Vv V WV 


solving complex real-world problems 


Gains in Personal and Social Development 


>» developing a personal code of values and ethics 

> understanding yourself 

>» understanding people of other racial and ethnic backgrounds 
» voting in local, state, or national elections 
>» learning effectively on your own 


Vv 


contributing to the welfare of your community 


>» developing a deepened sense of spirituality 


3 


National Survey of Student Engagement, “Creating Scales and Scalelets,” accessed May 30, 2013, http://nsse.iub.edu/_/?cid=368. 
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Student Participation in High-Impact Practices 


The data set used for Part I of Assessing Underserved Students’ Engagement in High-Impact Practices 
included National Survey of Student Engagement data from students at thirty-eight institutions 
across the state higher education systems in California, Oregon, and Wisconsin. The table below 
reflects the total number of students in the sample who participated in cumulative high-impact 
practices (HIPs) by class rank, gender, and underserved category. 


First-Year 


Sophomore 


Junior 


Senior 


Unclassified 


Female | 5,165 7,496 
Male | 3,209 4,476 


African American 


Hispanic 


Asian American 


Other 


No Response 


ans Oy 1836 / 248 | 4 611 22 7 


| Pavan 
| azar aL | 1,055 


Not First-Generation 3,781 5,821 2,310 | 395 
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Focus Group Script 


First, thank you for taking the time to come this afternoon. As you read in your invitation, this is one of 
several meetings being held nationally with students to learn more about your experiences in college. 
The reason we are here today is to better understand the types of learning experiences you’ve been 
engaged in during your time at [institution name], how those experiences have affected your learning, 
and how those experiences have affected you as a person. We promise to only take about an hour and a 
half of your time this afternoon. 


| also want to assure you that your names will not be disclosed or identified in later reports. We are only 
interested in getting your comments as a group. No individual names will in any way be connected to 
the comments you provide during our discussion. 


To assure accurate representation and reporting of our discussion later on, we will be video-recording 

and audio-recording our conversation. If you do not want to be video-recorded, you have the option of 
not participating in the focus group. The videos may be made public during AAC&U presentations and 
on the association’s website. 


Are there any questions before we start? 


Outline of Questions and Probes: 
Before we ask about your experiences as a student, wed like to get your opinion on what 
college in general means to you. 


1. In your opinion, what does a college education mean for individuals? In what ways does it 
matter to a person’s future? 


2. What do you believe potential employers are looking for in college graduates? 


a. What are the specific skills that you are learning (or hope to learn) in college that are 
important in the professional world? 


3. How well do you think your high school education prepared you to succeed in college? 


4. Could your high school (or schools) have done anything to better prepare you or your 
peers for college? 


5. Thinking about your experiences as a student, how would you describe what it means to 
be engaged in your learning? 


a. How do you know when you're engaged in learning versus simply learning? 
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6. Students often learn better in particular types of environments or doing particular types of 
activities. In your college experience so far, have there been certain activities or situations 
(inside or outside the classroom) that allowed you to be more engaged in your learning? If 
so, please describe. 


7. You all share in common that you have participated in particular types of learning activities or 
programs, such as [to be identified depending upon group]. How would you describe the ways 


in which being involved in one or more of these activities had an impact on your learning? 


a. How does this type of learning experience compare with other kinds of learning 
experiences you’ve had in college? 


b. What would you have changed about that experience to make it more engaging? 


8. In what ways did this experience influence the ways in which you interacted with the 
people around you? 


a. For example, how did this experience shape your interactions with peers? 
b. What about with faculty? 
c. What about people in the community (if applicable)? 


9. What did you learn about yourself through participating in these learning activities 
or programs? 


a. What more did you learn about your peers? 


b. In what ways did this experience have an impact on your understanding of the 
community or the larger world? 


10. What would you say most influences your decision to seek out and participate in specific 
types of engaged learning experiences—for example, service learning, undergraduate 
research (research with a faculty member), study abroad, or internships? 


11. In what ways have these types of learning experiences encouraged you to think differently 
about what you might do on campus or even after you leave campus? 


a. Have these experiences had any influence in your interests or goals, short term or long term? 


12. Has your engagement in these activities contributed to your social and ethical 
development? 


a. In what ways has your college experience prepared you to be a responsible and 
contributing member of your community? 


Finally, thinking about your view of college overall [Facilitators provide students with a 
handout with two statements]: 

This sheet lists two different views on the primary purpose and goal of a college education. Please 
read each statement and decide whether you agree with one of these statements, neither, or both. 


View A: The most important goal of a college education should be to provide students with a broad, 
well-rounded education that enriches them to discover their interests and abilities, in order to help them 
realize their full potential in life. 
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View B: The most important goal of a college education should be to provide students with specific career 
knowledge and skills to help them realize their full potential in the workforce. 


Probe: Which of these statements would you say comes closest to describing the emphasis of 


your college education thus far? 


Ending the session 


Is there anything that you feel we have missed or final comments you would like to add? 


Thank you all again very much for your time. And in case after you leave you have any additional thoughts 
or questions about our discussion, feel free to email us at: (facilitators give AAC&U e-mail addresses) 
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AAC&U Is THE LEADING NATIONAL ASSOCIATION concerned with the quality, 
vitality, and public standing of undergraduate liberal education. 

Its members are committed to extending the advantages of a liberal 
education to all students, regardless of academic specialization or intended 
career. Founded in1915, AAC&U now comprises nearly 1,300 member 
institutions—including accredited public and private colleges, community 
colleges, research universities, and comprehensive universities of every 


type and size. 


AAC&U FUNCTIONS AS A CATALYST AND FACILITATOR, forging links among 
presidents, administrators, and faculty members who are engaged in 
institutional and curricular planning. Its mission is to reinforce the 
collective commitment to liberal education and inclusive excellence at 
both the national and local levels, and to help individual institutions keep 
the quality of student learning at the core of their work as they evolve to 


meet new economic and social challenges. 


INFORMATION ABOUT AAC&U MEMBERSHIP, PROGRAMS, AND PUBLICATIONS 


CAN BE FOUND AT WWW. AACU.ORG. 


